Re: [PR] Fix non-deterministic iteration in SessionStateBuilder [datafusion]

["ahmed-mez (via GitHub)" <gi...@apache.org>]

ahmed-mez commented on code in PR #21262:
URL: https://github.com/apache/datafusion/pull/21262#discussion_r3052789266


##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -975,6 +975,20 @@ impl SessionState {
     }
 }
 
+/// Deduplicates function-registry map entries by keeping only entries whose key
+/// matches the canonical name. The session stores one hash map entry per alias
+/// plus the canonical name; filtering to canonical-name entries yields exactly
+/// one [`Arc`] per logical function.
+fn dedup_function_registry_by_canonical_name<T>(

Review Comment:
   Alternatively, it might be more robust to dedup by identity
   ```rust
     fn dedup_by_identity<T>(map: HashMap<String, Arc<T>>) -> Vec<Arc<T>> {
         let mut seen = HashSet::new();
         map.into_values()
             .filter(|arc| seen.insert(Arc::as_ptr(arc)))
             .collect()
     }
   ```



##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -975,6 +975,20 @@ impl SessionState {
     }
 }
 
+/// Deduplicates function-registry map entries by keeping only entries whose key
+/// matches the canonical name. The session stores one hash map entry per alias
+/// plus the canonical name; filtering to canonical-name entries yields exactly
+/// one [`Arc`] per logical function.
+fn dedup_function_registry_by_canonical_name<T>(

Review Comment:
   minor: I think the function could consume the map instead of borrowing it, to avoid some arc clones
   ```rust
     fn dedup_function_registry_by_canonical_name<T>(
         map: HashMap<String, Arc<T>>,
         canonical_name: impl Fn(&T) -> &str,
     ) -> Vec<Arc<T>> {
         map.into_iter()
             .filter(|(key, udf)| key.as_str() == canonical_name(udf.as_ref()))
             .map(|(_, udf)| udf)  // no Arc::clone needed
             .collect()
     }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org