From 4bd65b9f71d85f7824b2c2334ca8ac6ca58f9ee8 Mon Sep 17 00:00:00 2001 From: Jim Ingham Date: Wed, 18 Mar 2015 23:16:58 +0000 Subject: [PATCH] Comment discouraging writing command based test cases. llvm-svn: 232689 --- lldb/test/README-TestSuite | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/lldb/test/README-TestSuite b/lldb/test/README-TestSuite index eba34e6..6df4d7b 100644 --- a/lldb/test/README-TestSuite +++ b/lldb/test/README-TestSuite @@ -133,3 +133,27 @@ $ DOTEST_PROFILE=YES DOTEST_SCRIPT_DIR=/Volumes/data/lldb/svn/trunk/test /System After that, I used the pstats.py module to browse the statistics: $ python /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/pstats.py my.profile + +o Writing test cases: + + We strongly prefer writing test cases using the SB API's rather than the runCmd & expect. + Unless you are actually testing some feature of the command line, please don't write + command based tests. For historical reasons there are plenty of examples of tests in the + test suite that use runCmd where they shouldn't, but don't copy them, copy the plenty that + do use the SB API's instead. + + The reason for this is that our policy is that we will maintain compatibility with the + SB API's. But we don't make any similar guarantee about the details of command result format. + If your test is using the command line, it is going to have to check against the command result + text, and you either end up writing your check pattern by checking as little as possible so + you won't be exposed to random changes in the text; in which case you can end up missing some + failure, or you test too much and it means irrelevant changes break your tests. + + However, if you use the Python API's it is possible to check all the results you want + to check in a very explicit way, which makes the tests much more robust. + + Even if you are testing that a command-line command does some specific thing, it is still + better in general to use the SB API's to drive to the point where you want to run the test, + then use SBInterpreter::HandleCommand to run the command. You get the full result text + from the command in the command return object, and all the part where you are driving the + debugger to the point you want to test will be more robust. -- 2.7.4