toke.c: Use UTF-8 macros for portability
authorKarl Williamson <public@khwilliamson.com>
Thu, 8 Nov 2012 19:02:29 +0000 (12:02 -0700)
committerKarl Williamson <public@khwilliamson.com>
Sun, 11 Nov 2012 18:50:11 +0000 (11:50 -0700)
commit54d004e86b75a3ee498f929d981dce174ce3b79a
tree49a03db98c4a1417eb3cc879ade67ef8f8bd8999
parent4ab10950b702d3352c12c94c5c910a0f5cc1ca97
toke.c: Use UTF-8 macros for portability

I noticed this code in toke.c that hand-rolled its own UTF-8 handling,
and is assuming an ASCII platform.  There are macros that do the same
thing that this code does, and are portable.
toke.c