Add unittest

Add unittest

[   36s] + ./test
[   36s]
[   36s] running 36 tests
[   37s] test default_tokenstream_is_empty ... ok
[   37s] test byte_order_mark ... ok
[   37s] test ident_empty - should panic ... ok
[   37s] test ident_invalid - should panic ... ok
[   37s] test ident_number - should panic ... ok
[   37s] test fail ... ok
[   37s] test ident_raw_underscore - should panic ... ok
[   37s] test ident_raw_reserved - should panic ... ok
[   37s] test joint_last_token ... ok
[   37s] test idents ... ok
[   37s] test lifetime_empty - should panic ... ok
[   37s] test lifetime_invalid ... ok
[   37s] test lifetime_number - should panic ... ok
[   37s] test literal_byte_string ... ok
[   37s] test literal_character ... ok
[   37s] test literal_float ... ok
[   37s] test literal_integer ... ok
[   37s] test literal_iter_negative ... ok
[   37s] test literal_parse ... ok
[   37s] test literal_raw_string ... ok
[   37s] test literal_string ... ok
[   37s] test literal_suffix ... ok
[   37s] test no_panic ... ok
[   37s] test punct_before_comment ... ok
[   37s] test non_ascii_tokens ... ok
[   37s] test raw_ident_invalid - should panic ... ok
[   37s] test raw_ident_number - should panic ... ok
[   37s] test raw_identifier ... ok
[   37s] test raw_idents ... ok
[   37s] test roundtrip ... ok
[   37s] test test_debug_ident ... ok
[   37s] test raw_ident_empty - should panic ... ok
[   37s] test tokenstream_size_hint ... ok
[   37s] test tuple_indexing ... ok
[   37s] test span_test ... ok
[   37s] test test_debug_tokenstream ... ok
[   37s]
[   37s] test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s
[   37s]
[   37s] + for unit_test in '${unit_test_list}'
[   37s] + ./comments
[   37s]
[   37s] running 4 tests
[   37s] test closed_immediately ... ok
[   37s] test incomplete ... ok
[   37s] test carriage_return ... ok
[   37s] test lit ... ok
[   37s]
[   37s] test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[   37s]
[   37s] + for unit_test in '${unit_test_list}'
[   37s] + ./marker
[   37s]
[   37s] running 20 tests
[   37s] test Delimiter ... ok
[   37s] test Group ... ok
[   37s] test Ident ... ok
[   37s] test LexError ... ok
[   37s] test Literal ... ok
[   37s] test Punct ... ok
[   37s] test Spacing ... ok
[   37s] test Span ... ok
[   37s] test TokenStream ... ok
[   37s] test TokenTree ... ok
[   37s] test unwind_safe::Delimiter ... ok
[   37s] test unwind_safe::Group ... ok
[   37s] test unwind_safe::Ident ... ok
[   37s] test unwind_safe::LexError ... ok
[   37s] test unwind_safe::Punct ... ok
[   37s] test unwind_safe::Literal ... ok
[   37s] test unwind_safe::Spacing ... ok
[   37s] test unwind_safe::Span ... ok
[   37s] test unwind_safe::TokenStream ... ok
[   37s] test unwind_safe::TokenTree ... ok
[   37s]
[   37s] test result: ok. 20 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
[   37s]
[   37s] + for unit_test in '${unit_test_list}'
[   37s] + ./features
[   37s]
[   37s] running 1 test
[   37s] test make_sure_no_proc_macro ... ignored
[   37s]
[   37s] test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s