lib/math/rational.c: fix possible incorrect result from rational fractions helper