[I] Add `Metadata::retain` to filter entries in place [arrow-rs]

["alamb (via GitHub)" <gi...@apache.org>]

alamb opened a new issue, #10683:
URL: https://github.com/apache/arrow-rs/issues/10683

   **Is your feature request related to a problem or challenge?**
   
   While updating DataFusion to arrow 60 (https://github.com/apache/datafusion/pull/24366/changes), we found `Metadata` has no way to remove entries based on a predicate. Code that previously called `HashMap::retain` on field/schema metadata now has to convert to a `BTreeMap`, filter, and convert back.
   
   For example, DataFusion computes the intersection of metadata across UNION inputs, which now looks like:
   
   ```rust
   let mut intersected: Option<BTreeMap<String, String>> = None;
   
   for metadata in metadatas {
       match &mut intersected {
           None => {
               // deep copy into a BTreeMap so we can filter it below
               intersected =
                   Some(metadata.iter().map(|(k, v)| (k.clone(), v.clone())).collect());
           }
           Some(current) => {
               current.retain(|k, v| metadata.get(k) == Some(&*v));
           }
       }
   }
   
   intersected.map(Metadata::from).unwrap_or_default()
   ```
   
   **Describe the solution you'd like**
   
   A `Metadata::retain(|k, v| ...)` method, copy-on-write like the existing `insert`/`remove`. The code above would then stay in terms of `Metadata` and avoid the deep copy and round-trip:
   
   ```rust
   let mut intersected: Option<Metadata> = None;
   
   for metadata in metadatas {
       match &mut intersected {
           None => intersected = Some(metadata.clone()), // cheap clone
           Some(current) => current.retain(|k, v| metadata.get(k) == Some(&*v)),
       }
   }
   
   intersected.unwrap_or_default()
   ```
   
   **Describe alternatives you've considered**
   
   Converting through `BTreeMap` as shown above; it works but adds a copy and obscures the intent.
   
   **Additional context**
   
   Found while testing DataFusion against arrow-rs main ahead of the 60.0.0 release: https://github.com/apache/datafusion/pull/24366/changes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add `Metadata::retain` to filter entries in place [arrow-rs]

["alamb (via GitHub)" <gi...@apache.org>]

alamb closed issue #10683: Add `Metadata::retain` to filter entries in place
URL: https://github.com/apache/arrow-rs/issues/10683


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org