mesa/vbo: Fix scaling issue in 10-bit signed normalized packing.
For the 10-bit components, the divisor was incorrect. A 10-bit signed
integer can represent -2^9 through 2^9 - 1, which leads to the following
ranges:
(float)value.x -> [ -512, 511]
2.0F * (float)value.x -> [-1024, 1022]
2.0F * (float)value.x + 1.0F -> [-1023, 1023]
So dividing by 511 would incorrectly scale it to approximately:
[-2.
001956947, 2.
001956947]. To correctly scale to [-1.0, 1.0], we need
to divide by 1023.
This correctly implements the desktop GL rules. ES 3.0 has different
rules, but those will be implemented in a separate patch.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Marek Olšák <maraeo@gmail.com>