You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Jefffrey (via GitHub)" <gi...@apache.org> on 2026/01/25 03:32:26 UTC
[I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Jefffrey opened a new issue, #19982:
URL: https://github.com/apache/datafusion/issues/19982
### Describe the bug
https://github.com/apache/datafusion/blob/e5e76366a6200dcd7b85cc030128a191e3763dd1/datafusion/expr/src/udf.rs#L401-L419
The `ScalarValue`s in `scalar_arguments` don't necessarily have types which match `arg_fields` which can lead to confusion and extra handling in UDF definitions.
### To Reproduce
```rust
// datafusion/core/tests/user_defined/user_defined_scalar_functions.rs
#[tokio::test]
async fn test_scalar_arg_types() -> Result<()> {
#[derive(Debug, PartialEq, Eq, Hash)]
struct TestUdf {
signature: Signature,
}
impl Default for TestUdf {
fn default() -> Self {
Self {
signature: Signature::coercible(
vec![Coercion::new_implicit(
TypeSignatureClass::Native(logical_int16()),
vec![TypeSignatureClass::Numeric],
NativeType::Int16,
)],
Volatility::Immutable,
),
}
}
}
impl ScalarUDFImpl for TestUdf {
fn as_any(&self) -> &dyn Any {
self
}
fn name(&self) -> &str {
"test_udf"
}
fn signature(&self) -> &Signature {
&self.signature
}
fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> {
unreachable!()
}
fn return_field_from_args(&self, args: ReturnFieldArgs) -> Result<FieldRef> {
assert_eq!(args.arg_fields.len(), 1);
assert_eq!(args.scalar_arguments.len(), 1);
assert_eq!(
args.arg_fields[0].data_type(),
&args.scalar_arguments[0].unwrap().data_type()
); // <-- This assert failing
Ok(
Field::new(self.name(), args.arg_fields[0].data_type().clone(), true)
.into(),
)
}
fn invoke_with_args(&self, args: ScalarFunctionArgs) -> Result<ColumnarValue> {
assert!(matches!(
args.args[0],
ColumnarValue::Scalar(ScalarValue::Int16(Some(_)))
));
Ok(args.args[0].clone())
}
}
let ctx = SessionContext::new();
ctx.register_udf(TestUdf::default().into());
ctx.sql("select test_udf(1)").await?.collect().await?;
Ok(())
}
```
Running this test:
```sh
datafusion (main)$ cargo test -p datafusion --test user_defined_integration user_defined::user_defined_scalar_functions::test_scalar_arg_types -- --nocapture --exact
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.38s
Running tests/user_defined_integration.rs (/Users/jeffrey/.cargo_target_cache/debug/deps/user_defined_integration-53cd66eeac0051d3)
running 1 test
thread 'user_defined::user_defined_scalar_functions::test_scalar_arg_types' (14600918) panicked at datafusion/core/tests/user_defined/user_defined_scalar_functions.rs:2264:13:
assertion `left == right` failed
left: Int16
right: Int64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test user_defined::user_defined_scalar_functions::test_scalar_arg_types ... FAILED
failures:
failures:
user_defined::user_defined_scalar_functions::test_scalar_arg_types
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 79 filtered out; finished in 0.01s
error: test failed, to rerun pass `-p datafusion --test user_defined_integration`
```
The field says the argument should be an i16 according to signature, but the scalar argument is a i64.
### Expected behavior
Either somehow get type coercion to apply on `scalar_arguments` (not sure if this is possible), or update documentation to make this limitation clear.
### Additional context
Related code
https://github.com/apache/datafusion/blob/e5e76366a6200dcd7b85cc030128a191e3763dd1/datafusion/expr/src/expr_schema.rs#L576-L614
Might also affect udaf/udwf?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal commented on issue #19982:
URL: https://github.com/apache/datafusion/issues/19982#issuecomment-4505039877
```
cargo test -p datafusion --test user_defined_integration user_defined::user_defined_scalar_functions::test_scalar_arg_types -- --nocapture --exact
Finished [`test` profile [unoptimized + debuginfo]](https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles) target(s) in 2m 50s
Running tests/user_defined_integration.rs (/Users/kumarujjawal/.cargo/targets/stable-aarch64-apple-darwin/debug/deps/user_defined_integration-89174e31463cb665)
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 79 filtered out; finished in 0.00s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on issue #19982:
URL: https://github.com/apache/datafusion/issues/19982#issuecomment-3830962117
Thanks for confirming this 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "Trikooo (via GitHub)" <gi...@apache.org>.
Trikooo commented on issue #19982:
URL: https://github.com/apache/datafusion/issues/19982#issuecomment-3796429610
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "Trikooo (via GitHub)" <gi...@apache.org>.
Trikooo commented on issue #19982:
URL: https://github.com/apache/datafusion/issues/19982#issuecomment-3830665932
I looked into whether this affects UDAFs/UDWFs. It seems that it doesn't. Neither of them use `ReturnFieldArgs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "Trikooo (via GitHub)" <gi...@apache.org>.
Trikooo commented on issue #19982:
URL: https://github.com/apache/datafusion/issues/19982#issuecomment-4168390312
untake
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields` [datafusion]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey closed issue #19982: `ReturnFieldArgs.scalar_arguments` type doesn't match with `arg_fields`
URL: https://github.com/apache/datafusion/issues/19982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org