selftests: Improve reporting of problems with loading expected output

If the expected output file was missing (e.g. not included in
selftests.qrc), tst_selftests would trigger an assert inside QList by
calling QList::at() on an empty list.  Make tst_selftests detect this
error instead and give a meaningful error message.

When loading expected output for the crashes selftest, where there are
several alternative versions of the expected output, the code reused the
"exp" variable when loading the alternative test output files.  This
caused the last file loaded to be used unintentionally if none of the
alternative files had the correct number of lines.  Use a different
variable so that exp remains empty if none of the alternatives are
valid and a failure can be reported.

Change-Id: I35b2a3d905d069d3ee8dcb1447836eb68d5c8612
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>

diff --git a/tests/auto/testlib/selftests/tst_selftests.cpp b/tests/auto/testlib/selftests/tst_selftests.cpp
index a9ec4e3..97458ab 100644 (file)
--- a/tests/auto/testlib/selftests/tst_selftests.cpp
+++ b/tests/auto/testlib/selftests/tst_selftests.cpp
@@ -571,12 +571,13 @@ void tst_Selftests::doRunSubTest(QString const& subdir, QStringList const& logge
         // the actual output.
         if (exp.count() == 0) {
             QList<QList<QByteArray> > expArr;
+            QList<QByteArray> tmp;
             int i = 1;
             do {
-                exp = expectedResult(subdir + QString("_%1").arg(i++), logger);
-                if (exp.count())
-                    expArr += exp;
-            } while (exp.count());
+                tmp = expectedResult(subdir + QString("_%1").arg(i++), logger);
+                if (tmp.count())
+                    expArr += tmp;
+            } while (tmp.count());
 
             for (int j = 0; j < expArr.count(); ++j) {
                 if (res.count() == expArr.at(j).count()) {
@@ -584,12 +585,24 @@ void tst_Selftests::doRunSubTest(QString const& subdir, QStringList const& logge
                     break;
                 }
             }
+
+            if (expArr.count()) {
+                QVERIFY2(exp.count(),
+                         qPrintable(QString::fromLatin1("None of the expected output files for "
+                                                        "%1 format has matching line count.")
+                                    .arg(loggers.at(n))));
+            }
         } else {
             QVERIFY2(res.count() == exp.count(),
                      qPrintable(QString::fromLatin1("Mismatch in line count: %1 != %2 (%3).")
                                 .arg(res.count()).arg(exp.count()).arg(loggers.at(n))));
         }
 
+        // By this point, we should have loaded a non-empty expected data file.
+        QVERIFY2(exp.count(),
+                 qPrintable(QString::fromLatin1("Expected test data for %1 format is empty or not found.")
+                            .arg(loggers.at(n))));
+
         // For xml output formats, verify that the log is valid XML.
         if (logFormat(logger) == "xunitxml" || logFormat(logger) == "xml" || logFormat(logger) == "lightxml") {
             QByteArray xml(actualOutputs[n]);