toke.c: Remove character set dependency
authorKarl Williamson <public@khwilliamson.com>
Mon, 4 Mar 2013 16:14:25 +0000 (09:14 -0700)
committerKarl Williamson <public@khwilliamson.com>
Thu, 29 Aug 2013 15:56:00 +0000 (09:56 -0600)
commit8c4a053775b71d5f962f7a584bc3d44b0092d9b2
tree4d3e7106ab8e7c950861785723d36e061bd87a17
parent5f0aa340d4f4312ffa23535d638d891e4f3b2f7a
toke.c: Remove character set dependency

Instead of hard-coding the bit patterns that comprise the Byte Order
Mark in the UTF-8 or UTF-EBCDIC encodings, use the generated ones for
the current platform.

This removes some EBCDIC-only code.
toke.c