[I] `approx_percentile_cont` with a large `centroids` argument panics with "capacity overflow" [datafusion]

["edubraqd (via GitHub)" <gi...@apache.org>]

edubraqd opened a new issue, #24900:
URL: https://github.com/apache/datafusion/issues/24900

   ### Describe the bug
   
   `TDigest::merge_sorted_f64` and `TDigest::merge_digests` (`datafusion/functions-aggregate-common/src/tdigest.rs`) allocate the compressed centroid list with `Vec::with_capacity(max_size)`. `max_size` is the user supplied `centroids` argument of `approx_percentile_cont`, which `validate_input_max_size_expr` accepts for any positive integer. A large value therefore panics inside `Vec::with_capacity` (or aborts the process on allocation failure for values that fit but are still huge).
   
   ### To Reproduce
   
   ```sql
   SELECT approx_percentile_cont(x, 0.5, 9223372036854775807) FROM (VALUES (1), (2)) t(x);
   ```
   
   ```text
   thread 'main' panicked at library/alloc/src/raw_vec/mod.rs:28:5:
   capacity overflow
   ```
   
   ### Expected behavior
   
   The query returns a result. The compressed digest can never hold more centroids than the inputs being merged, so the allocation should be bounded by the input size rather than by `max_size` alone.
   
   ### Additional context
   
   Found while running a corpus of extreme-value literals against a debug build of `datafusion-cli`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] `approx_percentile_cont` with a large `centroids` argument panics with "capacity overflow" [datafusion]

["nessaisling-lab (via GitHub)" <gi...@apache.org>]

nessaisling-lab commented on issue #24900:
URL: https://github.com/apache/datafusion/issues/24900#issuecomment-6484583602

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org