Thursday, August 29, 2013

Create Test Suite for Tycho after login

Sometimes tests fail when they are not actually running. This typically happens in case of integration-tests. When something goes wrong(like login failed) the consequent tests all get failed and the build server starts flooding with the infamous All tests failed! mails!

What to do in such case?
Should we run the tests if the login failed even if they were supposed to run after a successful login?

We faced this recently with our RCP application which connects to a remote server to properly function.
We have around 50 SWTBot ui-tests for testing the features. The tests are run in our CI server.
SWTBot first logs onto the server using a login dialog.
But when the login fails we don't want to run the tests at all, here's what we did:

We created a TestSuite which returns all tests only when login succeeds.
Of course it can be customized further to give more logic to load each test case (like different test cases for different platforms!)

package package.of.test.suite;

import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;

import org.apache.log4j.Logger;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;

/*
 * A test suite which creates tests depending on whether the login succeeded or not.
 * Return all tests if login succeeds
 * Else return a suite containing no tests.
 */
public class AllTests extends TestSuite {

    private static boolean loggedIn = initializeProductForTesting();
    private static Logger  mLogger  = Logger.getLogger(AllTests.class);
    private static SWTWorkbenchBot bot;
    public static boolean initializeProductForTesting() {
        System.out.println("in Beforeclass method before all tests!");
        Utils.login();
        return Utils.isLoggedIn();
    }

    public static TestSuite suite() {
        TestSuite suite = new TestSuite();
        if (loggedIn) {
            suite.addTest(new JUnit4TestAdapter(Test1.class));
            suite.addTest(new JUnit4TestAdapter(Test2.class));
            suite.addTest(new JUnit4TestAdapter(Test3.class));
        }
        else {
            mLogger.debug("Can't log into server!");
            mLogger.debug("No tests will run !");
            Display.getDefault().close();
            System.exit(0);
        }
        return suite;
    }
}

We configure tycho-surefire plugin to run this suite
<configuration>
                    ...
                    <testSuite>bundle.symbolic.name.of.test.plugin</testSuite>
                    <testClass>package.of.test.suite.AllTests</testClass>
                    ...
</configuration>

No comments:

Post a Comment