[flang] runtime: For Fw.d formatting, don't oscillate forever
authorpeter klausler <pklausler@nvidia.com>
Mon, 2 Aug 2021 21:37:40 +0000 (14:37 -0700)
committerpeter klausler <pklausler@nvidia.com>
Wed, 4 Aug 2021 19:19:23 +0000 (12:19 -0700)
commit4876520eef34f517075fed9007dc8b3162d67d1a
tree5125481fabd927bdf439b6aaa2cdc6a5145cd2f6
parent617be2756fd0e0d943d082e8f86309c4133ce64b
[flang] runtime: For Fw.d formatting, don't oscillate forever

The algorithm for Fw.d output will drive binary to decimal conversion for
an initial fixed number of digits, then adjust that number based on the
result's exposent.  For value close to a power of ten, this adjustment
process wouldn't terminate; e.g., formatting 9.999 as F10.2 would start
with 1e2, boost the digits to 2, get 9.99e1, decrease the digits, and loop.
Solve by refusing to boost the digits a second time.

Differential Revision: https://reviews.llvm.org/D107490
flang/lib/Decimal/binary-to-decimal.cpp
flang/runtime/edit-output.cpp
flang/unittests/Runtime/NumericalFormatTest.cpp