fix RT#114068 optimizer handles MEOL in middle of pattern improperly
It seems that under certain circumstances the optimiser handles the
MEOL operator (what $ turns into under /m), improperly including
things that follow. This results in compilation like this:
Compiling REx "( [^z] $ [^z]+ )"
Final program:
1: OPEN1 (3)
3: ANYOF[\x00-y{-\xff][{unicode}0100-INFINITY] (14)
14: MEOL (15)
15: PLUS (27)
16: ANYOF[\x00-y{-\xff][{unicode}0100-INFINITY] (0)
27: CLOSE1 (29)
29: END (0)
anchored ""$ at 2 stclass ANYOF[\x00-y{-\xff][{unicode}0100-INFINITY]
Where the '""$ at 2' is sign of the bug. The problem is that the optimiser
does not "commit" the string when it encounters an MEOL, which means that
text that follows it is included. This has probably always been wrong as
$ is a multichar pattern (it matches before an \n or including an \n). This
failure to commit then interacts with the implementation for PLUS leading to
an incorrect offset. By adding a SCAN_COMMIT() as part of the optimisers
handling of EOL constructs this problem is avoided. Note that most uses of
$ were ok due to other reasons.