Fix x86 encoder to use 64-bit type to accumulate opcode/prefix bits

Fix x86 encoder to use 64-bit type to accumulate opcode/prefix bits

The encoder was using size_t, a 32-bit type on x86, to accumulate opcode
and prefix bits to emit. AVX support uses 3 bytes for prefixes that are
higher than the 32-bit type can handle. So, change all code byte related types
from size_t to a new code_t, defined as "unsigned __int64" on RyuJIT x86
(there is precedence for this type on the ARM architectures).

Fixes #8331