-Add an option (probably -L) for turning off line numbers for debugging the
-generated state machine.
+Ragel 6.0 Changes
+=================
-Expressions of the form: expr1 <: expr2 . expr3
-sometimes don't behave as expected. If expr2 contains the empty string then
-it's possible for the machine to escape to expr3. This is somewhat unexpected.
-The empty string does count as moving through the right machine so there should
-not be an exception for this case.
-Using the stronger implementation: ( expr1 $1 %0 . expr2 ) will solve the problem.
+Remove old action embedding and condition setting syntax.
+
+Expressions of the form: ( expr1 <: expr2 . expr3 ) sometimes don't behave as
+expected. If expr2 contains the empty string then it's possible for the machine
+to escape to expr3. This is somewhat unexpected. The empty string does count
+as moving through the right machine so there should not be an exception for
+this case. Using the stronger implementation: ( expr1 $1 %0 . expr2 ) will
+solve the problem.
Also, there is potential for nondeterminism to persist via the empty string
with the :> and :>> operators. Should also guard against this using leaving
priorities.
-The |> guarded operator and the <| guarded operator need to be added.
+fbreak should advance the current char. Depreciate fbreak and add
+ fctl_break;
+ fctl_return <expr>;
+ fctl_goto <label>;
+This is needed for better support of pull scanners.
+
+fbreak should execute the to-state actions of the target state. The semantics
+of fbreak should be identical to reaching the end of a buffer block.
+
+Introduce a new variable "eof" which is a pointer like pe. It gets set to pe
+when the input has ended. This would enable eof actions to go into the main
+loop, granting them the ability to manipulate p and cs. This would free us from
+requiring some eof character to be sent to scanners to 'flush out' the last
+token. Instead an eof action can now execute scanner pattern actions and
+backtrack if necessary.
+
+The tokstart/tokend variable should be shortened to ts/te.
+
+Eliminate tokend, replace it with the marker variable and add ftokend/ftoklen.
+
+Add the accept action embedding operator: like eof but only for states that end
+up final. Add the combined pending/accept action. This becomes a good idea when
+eof action execution is moved into the main loop.
+
+Include file searching should be relative to the source file. Should add a -I
+option.
+
+================
+END 6.0 Features
+
+Add a prefix operator which sets every state final.
+
+Minimization should remove a condition when the character allows both
+the positive and negative sense of the condition. This happens in:
+test_every_10_chars = ( ( c when test_len ) c{0,9} )**;
+In this example there is non-determinsm that is killed by the priorities, but
+since conditions are expanded before priorities are tested, many transitions
+end up with test_len || !test_len.
-The fixed size stack is a problem for manual recursion of unlimited depth.
+Should be possible to include scanner definitions in another scanner.
+
+Need an "entry name;" feature, causing name to get written out with the other
+entry points in the data.
+
+Possibly bring back the old semantics of > in a new operator, or allow it to be
+defined somehow.
+
+When priorities are embedded without a name, the name of the current machine is
+used. Perhaps a unique name for each instance of the current machine should be
+used instead. This idea could work well if applied to user-defined embeddings.
+
+User defined embeddings <-name(a1,a2,...).
+User defined operators expr1 <name> expr2.
+
+Doesn't make make sense for [] to be the lambda (zero-length) machine. This
+should be the empty set (the empty machine). But then would it be invalid in a
+regular expression?
+
+The |> guarded operator and the <| guarded operator need to be added.
An option to turn off the removal of duplicate actions might be useful for
analyzing unintentional nondeterminism.
-To avoid confusion, there should be a correspondence between state numbers in
-the generated code, graphviz and the XML.
-
Might be a good idea to add in some protection against using up all of a
system's memory. This protection could then be removed by people when someone
is sure they want to use a lot of memory.
-fbreak should advance the current char. Depreciate fbreak and add
- fctl_break;
- fctl_return <expr>;
- fctl_goto <label>;
-
-Should the fbreak functions execute the to-state actions of the target state?
-
-It should be possible to import/export definitions.
If a scanner can be optimized into a pure state machine, maybe permit it to be
referenced as a machine definition. Alternately: inline scanners with an
projects / external / ragel.git / blobdiff