Update tests README

diff --git a/tests/README.md b/tests/README.md
index 6367c629cee6fb59b1cea4526db6494f86e83195..2f4da6fb2b0491b1873e52c7efc05738f2e96100 100644 (file)
--- a/tests/README.md
+++ b/tests/README.md
@@ -1,75 +1,109 @@
-# netcoregbd tests
+# Tests
 
 ## Running tests
 
+Before running tests the debugger should be built and installed inside `./bin` directory, `dotnet` CLI shoud be present in `PATH`.
+
+From project root directory test are run through dotnet CLI:
 ```
-git clone https://github.sec.samsung.net/i-kulaychuk/netcoredbg.git
-cd netcoredbg
+cd tests
 dotnet test
 ```
 
-1. To run tests with dotnet:
-
+It is possible to override path to the debugger binary through `PIPE` environgment variable:
 ```
-CORERUN="/path/to/corerun" DEBUGGER="/path/to/debugger" dotnet test
+cd tests
+PIPE=<path-to-debugger> dotnet test --logger "trx;LogFileName=$PWD/Results.trx"
 ```
 
-2. To run tests with corerun:
-After ```dotnet build``` nuget packages will be downloaded into ~/.nuget
-To run tests with corerun, probably, you should copy some libraries from nuget packages into folder with corerun, like Microsoft.CodeAnalysis.*.dll and xunit.assert.dll (corerun will say if it miss some library).
+`PIPE` can contain multiple shell commands, but in the end the debugger MI console should be lanched.
 
+Test results can be exported to a file with `--logger` command line option.
+
+By default test binaries are descovered recursively under project `tests` directory. When tests should be executed on another machine, environment variable `TESTDIR` can override the location of test binaries. In that case recursive search is not performed and all test binaries should be located directly under `TESTDIR`. So for `TESTDIR=/tmp` the test binary files layout shoud be:
 ```
-dotnet build
-CORERUN="/path/to/corerun" DEBUGGER="/path/to/debugger" /path/to/corerun /path/to/launcher.dll
+/tmp/Example1Test.dll
+/tmp/Example1Test.pdb
+/tmp/Example2Test.dll
+/tmp/Example2Test.pdb
+...
 ```
 
-Result logs will appear in netcoredbg/tests/runner/bin/Debug/netcoreapp2.0/<test_start_date>
+Refer to `run_tests_sdb.sh` as an example of running tests on Tizen device.
+
+## Writing new tests
+
+### Add new test
+
+Create new project under `tests` directory and add it to `tests` solution:
+```
+cd tests
+dotnet new console -o ExampleTest
+dotnet sln add ExampleTest/ExampleTest.csproj
+```
 
-## Creating test steps
+Add project name as a method of `Runner.TestRunner` class in `runner/Runner.cs`:
+```
+[Fact]
+public void ExampleTest() => ExecuteTest();
+```
 
-- Сreate some folder for test case inside 'tests' dir.
-- Сreate 'test_case_name.csproj' file and run ```dotnet sln add /path/to/test_case_name.csproj```.
-- Сreate 'test_case_name.cs' file with test case as written below.
+Method name should match the project name and the name of the dll that is produced after project build.
 
-## Writing test case
+Now the ExampleTest should be displayed as available:
+```
+$ dotnet test -t
+...
+The following Tests are available:
+    ...
+    Runner.TestRunner.ExampleTest
+```
 
-1. See example test case inside 'tests/example' folder.
-1. Write 'test_case_name.cs' file, which will be launched under the debugger.
-1. Write scenario for debugger in comments inside 'test_case_name.cs' file.
+### Write test scenario
 
-### Writing scenario
+Test scenario should be present in comments inside TestExample/Program.cs
 
-You should write scenario for debugger inside comments in 'test_case_name.cs' file, this comments will be executed with Roslyn.
+The following example instructs the debugger to run test binary and verifies that the debugger stops at line with `@START@` tag with stop reason `entry-point-hit`:
+```
+using System;
+/*
+Send("1-file-exec-and-symbols dotnet");
+Send("2-exec-arguments " + TestBin);
+Send("3-exec-run");
+
+var r = Expect("*stopped");
+Assert.Equal(r.FindString("reason"), "entry-point-hit");
+Assert.Equal(r.Find("frame").FindInt("line"), Lines["START"]);
+*/
+namespace ExampleTest
+{
+    class Program
+    {
+        static void Main(string[] args)
+        { // //@START@
+            Console.WriteLine("Hello World!");
+        }
+    }
+}
+```
 
-#### Multiline comments
+#### Variables
 
-Don't use ```/**/``` for multiline comment. If you want to write comment on line with Roslyn commands - you should write it, using '/**/'.
-For multiline comments ```_currentLine``` and ```_commandLine``` variables will always be equal to number of first line.
+* `Dictionary<string, int> Lines` provides the mapping between tags `@TAGNAME@` in test source and line numbers.
 
-#### You must do
+* `string TestSource` is a full path to test source file.
 
-- Write ```// $Main$``` tag on line with curly bracket after 'Main()'. At the beginning of every test case debugger running to Main.
-- Write ```// start()``` on line, where you want to start test case. Debugger will run to line with this comment.
-- Write ```// send("-gdb-exit")``` to finish test case.
+* `string TestBin` is a full path to test dll file.
 
-#### Global test case funciotns
+* `ITestOutputHelper Output` is a log interface from Xunit, it contains methods like `WriteLine` etc.
 
-- ```MICore.ResultValue send(string cmd, int expectedLine = -1, int times = 1)``` - to send command 'cmd' to debugger. If command is '-exec-run', '-exec-continue', '-exec-step' or '-exec-next' - function will wait for the debugger to stop on 'expectedLine'. This function returns parsed respond: "\*stopped.*" for move commands and "\^done.*" for rest commands.
-- ```Match expect(string s)``` - to find line 's' in debugger output, using regexp and return 'RegularExpressions.Match'. If this function wont find match in 'EXPECT_TIMEOUT' seconds - it will throw exception.
-- ```void assertEqualRe(string expect, string actual, string msg = "", bool expectedRes = true)``` - to compare 'expect' and 'actual' strings with regexp and fail test, in case the result not equal to 'expectedRes'
+#### Methods
 
-#### Global test case variables
+* `Assert.*` are asserts from Xunit.
 
-- ```int _current_line``` - has value of current line of '.cs' file. Functions 'nextTo()', 'stepTo()' and 'breakTo()' changing value of this variable, so if you use several of this functions inside single comment - '_current_line' variable will change while the comment is being executed.
-- ```int _command_line``` - has value of current line of '.cs' file. Unlike '_current_line', '_command_line' will have same value during all comment.
-- ```Dictionary<string, int> Tags``` - Dictionary, which contains line numbers with tags.
-  - write ```// $ tag_name $``` to add line number to 'Tags' dictionary with 'tag_name' key.
+* `void Send(string s)` sends command to the debugger process through stdin. New line is automatically appended to `s`.
 
-#### Example
+* `MICore.Results Expect(string s, int timeoutSec = 10)` reads debugger output line by line and checks if some line starts with `s`. If the expected line is found then it is parsed into MI responce object `MICore.Results`. In case of timeout or invalid MI responce an exception is thrown.
+For more info about MI responce object see https://github.com/Microsoft/MIEngine/blob/master/src/MICore/MIResults.cs
 
-```
-int x = 13;            // send("-exec-step", expectedLine: Tags["MY_TAG1"]);
-Console.WriteLine(x);  // /* $ MY_TAG1 $ */
-                       // var r = send(@"-var-create - * ""x""");
-                       // assertEqualRe("13", r.Find("value").ToString());
-```
+* `int GetCurrentLine()` provides current line number.