[GitHub] [nuttx] xaowang96 opened a new pull request, #8356: libc/stdio: fix rounding errors for fractional values less than 1

["xaowang96 (via GitHub)" <gi...@apache.org>]

xaowang96 opened a new pull request, #8356:
URL: https://github.com/apache/nuttx/pull/8356

   Signed-off-by: wangxuedong <wa...@xiaomi.com>
   
   ## Summary
   
   Now all decimal values greater than 0.5 and less than 1 are converted to strings with wrong results. This PR is to solve this 
   problem.
   
   If we want to convert decimal 0.58322f to string with one significant digit to the right of the decimal point, like this
   ` snprintf(buff, 8, "%.*f", 1, 0.58322ff);`. In this case the conversion result will be 0.5 instead of the expected 0.6.
   
   The reason is because the calculate result of this code `max_digits = MIN(max_digits, max_decimals + MAX(exp + 1, 1));
   ` is incorrect. The calculated max_digits value is 2, and then 0.005 will be added here `x = x + g_dtoa_round[max_digits];`. After this, we get the result of 0.58822f which is wrong. To fix this error, we set max_digits to 1, 0.05 will be added to x, so we get the correct value 0.63322f.
   
   In the case of an integer value of 0,  there is only a decimal mantissa in dtoa->digits, and the value of max_digits should be consistent with the number of digits to the right of the decimal point.
   decimal point.
   
   ## Impact
   
   Conversion of decimal values in spring functions.
   
   ## Testing
   
   N/A
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nuttx] xiaoxiang781216 merged pull request #8356: libc/stdio: fix rounding errors for fractional values less than 1

["xiaoxiang781216 (via GitHub)" <gi...@apache.org>]

xiaoxiang781216 merged PR #8356:
URL: https://github.com/apache/nuttx/pull/8356


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org