Commit Graph

2 Commits

Author SHA1 Message Date
Dan Ellis
6f4d424f46 py/formatfloat: Use pow(10, e) instead of pos/neg_pow lookup tables.
Rework the conversion of floats to decimal strings so it aligns precisely
with the conversion of strings to floats in parsenum.c.  This is to avoid
rendering 1eX as 9.99999eX-1 etc.  This is achieved by removing the power-
of-10 tables and using pow() to compute the exponent directly, and that's
done efficiently by first estimating the power-of-10 exponent from the
power-of-2 exponent in the floating-point representation.

Code size is reduced by roughly 100 to 200 bytes by this commit.

Signed-off-by: Dan Ellis <dan.ellis@gmail.com>
2022-08-12 23:53:34 +10:00
Dan Ellis
f9cbe6bc47 py/formatfloat: Format all whole-number floats exactly.
Formerly, py/formatfloat would print whole numbers inaccurately with
nonzero digits beyond the decimal place.  This resulted from its strategy
of successive scaling of the argument by 0.1 which cannot be exactly
represented in floating point.  The change in this commit avoids scaling
until the value is smaller than 1, so all whole numbers print with zero
fractional part.

Fixes issue #4212.

Signed-off-by: Dan Ellis dan.ellis@gmail.com
2022-07-26 22:23:47 +10:00