You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/19 20:08:39 UTC
(datafusion) branch gh-readonly-queue/main/pr-25510-710c748db8931bbe28ade5f25436d0672a19b355 created (now 925d7f8ffd)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-25510-710c748db8931bbe28ade5f25436d0672a19b355
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 925d7f8ffd perf: Avoid needless conversion of `Metadata` to `HashMap` (#25510)
This branch includes the following new commits:
new 925d7f8ffd perf: Avoid needless conversion of `Metadata` to `HashMap` (#25510)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: perf: Avoid needless conversion of `Metadata` to `HashMap` (#25510)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25510-710c748db8931bbe28ade5f25436d0672a19b355
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 925d7f8ffdcb5584e26f2410facbcfc5b4234d20
Author: Neil Conway <ne...@gmail.com>
AuthorDate: Sat Sep 19 20:08:15 2026 +0000
perf: Avoid needless conversion of `Metadata` to `HashMap` (#25510)
## Which issue does this PR close?
- N/A
## Rationale for this change
Arrow 60 introduced a new `Metadata` type to represent field and schema
metadata. Iterating over a `Metadata` is guaranteed to proceed in
ascending key order.
This PR simplifies a few needless conversions and sorts when dealing
with `Metadata` values.
## What changes are included in this PR?
- `VALUES` schema builder: pass `Metadata` directly instead of
converting to a `HashMap`
- `arrow_metadata` and `arrow_field`: remove temporary vectors and
redundant sorting
- Add test to verify `arrow_field` returns metadata in ascending key
order
## What is the testing strategy for this PR?
Existing tests pass; new test added.
## Are there any user-facing changes?
No.
---
datafusion/expr/src/logical_plan/builder.rs | 2 +-
datafusion/functions/src/core/arrow_field.rs | 5 +----
datafusion/functions/src/core/arrow_metadata.rs | 5 +----
datafusion/sqllogictest/test_files/arrow_field.slt | 6 ++++++
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/datafusion/expr/src/logical_plan/builder.rs b/datafusion/expr/src/logical_plan/builder.rs
index a44505b15a..ba16116136 100644
--- a/datafusion/expr/src/logical_plan/builder.rs
+++ b/datafusion/expr/src/logical_plan/builder.rs
@@ -1710,7 +1710,7 @@ impl ValuesFields {
let name = format!("column{}", self.inner.len() + 1);
let mut field = Field::new(name, data_type, nullable);
if let Some(metadata) = metadata {
- field.set_metadata(metadata.to_hashmap());
+ field.set_metadata(metadata.into_inner());
}
self.inner.push(field);
}
diff --git a/datafusion/functions/src/core/arrow_field.rs b/datafusion/functions/src/core/arrow_field.rs
index 46b8e9872f..c6091f49fd 100644
--- a/datafusion/functions/src/core/arrow_field.rs
+++ b/datafusion/functions/src/core/arrow_field.rs
@@ -139,10 +139,7 @@ impl ScalarUDFImpl for ArrowFieldFunc {
StringBuilder::new(),
);
- let mut entries: Vec<_> = metadata.iter().collect();
- entries.sort_by_key(|(k, _)| *k);
-
- for (k, v) in entries {
+ for (k, v) in metadata {
map_builder.keys().append_value(k);
map_builder.values().append_value(v);
}
diff --git a/datafusion/functions/src/core/arrow_metadata.rs b/datafusion/functions/src/core/arrow_metadata.rs
index 37a2a52c19..8840e91270 100644
--- a/datafusion/functions/src/core/arrow_metadata.rs
+++ b/datafusion/functions/src/core/arrow_metadata.rs
@@ -142,10 +142,7 @@ impl ScalarUDFImpl for ArrowMetadataFunc {
StringBuilder::new(),
);
- let mut entries: Vec<_> = metadata.iter().collect();
- entries.sort_by_key(|(k, _)| *k);
-
- for (k, v) in entries {
+ for (k, v) in metadata {
map_builder.keys().append_value(k);
map_builder.values().append_value(v);
}
diff --git a/datafusion/sqllogictest/test_files/arrow_field.slt b/datafusion/sqllogictest/test_files/arrow_field.slt
index d33cd935bc..7c028c9703 100644
--- a/datafusion/sqllogictest/test_files/arrow_field.slt
+++ b/datafusion/sqllogictest/test_files/arrow_field.slt
@@ -134,6 +134,12 @@ SELECT arrow_field(id)['metadata'] FROM table_with_metadata LIMIT 1
----
{metadata_key: the id field}
+# Metadata entries are returned in key order, regardless of insertion order.
+query ?
+SELECT arrow_field(with_metadata(1, 'z', 'last', 'm', 'middle', 'a', 'first'))['metadata']
+----
+{a: first, m: middle, z: last}
+
# arrow_field nullability field access
query I
SELECT count(*) FROM table_with_metadata WHERE NOT arrow_field(id)['nullable']
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org