You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Druva-D (via GitHub)" <gi...@apache.org> on 2026/02/10 09:13:15 UTC

[I] format_string: grouping separator flag (,) ignored for float and decimal conversions (%f) [datafusion]

Druva-D opened a new issue, #20266:
URL: https://github.com/apache/datafusion/issues/20266

   ### Describe the bug
   
     The FormatStringFunc (datafusion/spark/src/function/string/format_string.rs) correctly parses the grouping separator flag (,) in format specifiers like %,.2f, setting 
   `ConversionSpecifier::grouping_separator =true` 
   However, the flag is only applied during integer formatting in format_unsigned() (for %d). It is silently ignored in two methods:
   
     1. format_float() (line ~1684) -- handles Float16, Float32, Float64 with %f, %e, %g conversions. The method never checks self.grouping_separator.
     2. format_decimal() (line ~2022) -- handles Decimal128, Decimal256 with %f, %e, %g conversions. Same issue.
   
     This is inconsistent with `java.util.Formatter` behavior, where %,.2f applies thousands separators to the integer portion of floating-point output.
   
   ### To Reproduce
   
   ```rs
   use datafusion::execution::SessionStateBuilder;
   use datafusion::prelude::SessionContext;
   use datafusion_spark::SessionStateBuilderSpark;
   
   #[tokio::main]
   async fn main() -> datafusion::common::Result<()> {
       let state = SessionStateBuilder::new()
           .with_default_features()
           .with_spark_features()
           .build();
       let ctx = SessionContext::new_with_state(state);
   
       let df = ctx
           .sql("SELECT format_string('%,.2f', 1234567.89)")
           .await?;
       df.show().await?;
   
       Ok(())
   }
   ```
   
   Output
   ```
   +--------------------------------------------------+
   | format_string(Utf8("%,.2f"),Float64(1234567.89)) |
   +--------------------------------------------------+
   | 1234567.89                                       |
   +--------------------------------------------------+
   +--------------------------------------------------+
   | format_string(Utf8("%,.2f"),Float64(1234567.89)) |
   +--------------------------------------------------+
   | 1234567.89                                       |
   +--------------------------------------------------+
   ```
   
   
   ### Expected behavior
   
   Got expected behaviour from Java
   ```java
   public class FormatTest {
       public static void main(String[] args) {
           // Float grouping separator
           System.out.println("Float %,.2f:  " + String.format("%,.2f", 1234567.89));
           System.out.println("Float %.2f:   " + String.format("%.2f", 1234567.89));
           System.out.println("Float %,.0f:  " + String.format("%,.0f", 1000.0));
   
           // Decimal (BigDecimal) grouping separator
           java.math.BigDecimal bd = new java.math.BigDecimal("1234567.89");
           System.out.println("BigDec %,.2f: " + String.format("%,.2f", bd));
           System.out.println("BigDec %.2f:  " + String.format("%.2f", bd));
       }
   }
   ```
   ```
   Float %,.2f:  1,234,567.89
   Float %.2f:   1234567.89
   Float %,.0f:  1,000
   BigDec %,.2f: 1,234,567.89
   BigDec %.2f:  1234567.89
   ```
   
   ### Additional context
   
   Apply the same thousands-separator loop already used in format_unsigned() to the integer part, then rejoin. This
     needs to be added in two places:
   
     - format_float(): after line ~1768 where number = format!("{abs:.prec$}", ...)
     - format_decimal(): after the number is produced from the match self.conversion_type block (~line 2095), before padding is applied
   
     Reference: the grouping logic already exists in format_unsigned() (line ~1897) and can be reused.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] format_string: grouping separator flag (,) ignored for float and decimal conversions (%f) [datafusion]

Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew closed issue #20266: format_string: grouping separator flag (,) ignored for float and decimal conversions (%f)
URL: https://github.com/apache/datafusion/issues/20266


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org