Self updating Eclipse RCP application with p2

We recently moved to self updating for our RCP application.
There is a nice wiki page on self-updating with several ways to achieve it.
We opted for the headless one as we wanted no user interaction at all.
Simple requirement:
During startup look for updates, if found, update.

Unfortunately the P2Util class in the wiki didn't work right away for our case.
So we made some modifications to make it work for us.

public class P2Util {
    private static final String JUSTUPDATED = "justUpdated";
    private final static org.apache.log4j.Logger mLogger = org.apache.log4j.Logger
.getLogger(P2Util.class.getName());

    public static void autoupdate(long millisDelay) {
        final IProvisioningAgent agent = (IProvisioningAgent) ServiceHelper.getService(Activator.getDefault()
                .getBundle().getBundleContext(), IProvisioningAgent.SERVICE_NAME);
        if (agent == null) {
            LogHelper.log(new Status(IStatus.ERROR, Activator.PLUGIN_ID,
                    "No provisioning agent found.  This application is not set up for updates."));
        }
        // XXX if we're restarting after updating, don't check again.
        final IPreferenceStore prefStore = Activator.getDefault().getPreferenceStore();
        if (prefStore.getBoolean(JUSTUPDATED)) {
            prefStore.setValue(JUSTUPDATED, false);
            return;
        }
        /*
         * We create a Job to run the update operation with a cancellable
         * dialog.
         */
        final Job job = new Job("Application Update") {

            @Override
            protected IStatus run(IProgressMonitor monitor) {
                IStatus updateStatus = update(agent, monitor);
                return updateStatus;
            }
        };
        job.setUser(true);
        /*
         * Hook after-update todos as a listener.
         */
        job.addJobChangeListener(new JobChangeAdapter() {
            public void done(IJobChangeEvent event) {
                IStatus updateStatus = event.getResult();
                if (updateStatus.getCode() == UpdateOperation.STATUS_NOTHING_TO_UPDATE) {
                    // do nothing..
                }
                else if (updateStatus.getSeverity() != IStatus.ERROR) {
                    prefStore.setValue(JUSTUPDATED, true);
                    Display.getDefault().syncExec(new Runnable() {

                        @Override
                        public void run() {
                            PlatformUI.getWorkbench().restart();
                        }
                    });
                }
                else {
                    mLogger.debug(updateStatus.getMessage());
                }
            }
        });
        job.schedule(millisDelay);

    }

    private static IStatus update(IProvisioningAgent agent, IProgressMonitor monitor) {
        UpdateOperation operation = null;
        /*
         * Any update has to be done against a profile. We'll use the first
         * profile in profile registry under which all the installable units are
         * registered. It's a self hosting profile.
         */
        IProfileRegistry registry = (IProfileRegistry) agent.getService(IProfileRegistry.SERVICE_NAME);
        IProfile[] profiles = registry.getProfiles();
        IProfile profile = null;
        if (profiles.length > 0) {
            profile = profiles[0];
        }
        else {
            return new Status(IStatus.ERROR, "", "No profile found to update!");
        }

        /*
         * Query to find all installable units in the profile. These will be
         * searched in the update site for updates.
         */
        Collection<iinstallableunit> toUpdate = profile.query(
QueryUtil.createIUAnyQuery(),
            monitor).toSet();
        ProvisioningSession provisioningSession = new ProvisioningSession(agent);
        operation = new UpdateOperation(provisioningSession, toUpdate);
        operation.setProfileId(profile.getProfileId());
        SubMonitor sub = SubMonitor.convert(monitor, "Checking for Application updates...", 200);
        IStatus status = operation.resolveModal(sub.newChild(100));
        if (status.getCode() == UpdateOperation.STATUS_NOTHING_TO_UPDATE) {
            return status;
        }
        if (status.getSeverity() == IStatus.CANCEL)
            throw new OperationCanceledException();

        if (status.getSeverity() != IStatus.ERROR) {
            // More complex status handling might include showing the user what
            // updates
            // are available if there are multiples, differentiating patches vs.
            // updates, etc.
            // We simply update as suggested by the operation without any user
            // interaction.
            sub.setTaskName("Updating Application to latest version available..");
            ProvisioningJob job = operation.getProvisioningJob(sub);
            status = job.runModal(sub.newChild(100));
            if (status.getSeverity() == IStatus.CANCEL)
                throw new OperationCanceledException();
        }
        return status;

    }

}

We add our update site to p2.inf file of the feature.

instructions.configure=\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:0,location:http${#58}//pc0020:8080/uploads/,name:Update Site );\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:1,location:http${#58}//pc0020:8080/uploads/,name:Update Site);\

Creating an excel (*.xls) file with apache.poi library

Apache poi provides a very clean api to create and manipulate microsoft documents.
Here an example of creating an xls file from a String having of format like below:
"column1";"column2";"column3" \n"line1value1";"line1value2";"line1value3"\n"line2value1";"line2value2";"line2value3"

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.hssf.util.HSSFColor;

public class StringToExcelFileConverter {

    public static File convert(String csvString) {

        try {

            File xlsTempFile = File.createTempFile("temp" + System.nanoTime(), ".xls");
            FileOutputStream fileOut = new FileOutputStream(xlsTempFile);
            HSSFWorkbook workbook = new HSSFWorkbook();
            HSSFSheet worksheet = workbook.createSheet("detail");
            String separator = getSeparator();//like ";"  in above string
            String[] rowStrings = csvString.split("\n");
            for (int i = 0; i < rowStrings.length; i++) {
                HSSFRow row1 = worksheet.createRow((short) i);
                String[] columns = rowStrings[i].split(separator);
                for (int j = 0; j < columns.length; j++) {
                    HSSFCell cell = row1.createCell((short) j);
                    cell.setCellValue(columns[j].replace("\"", ""));
                    HSSFCellStyle cellStyle = workbook.createCellStyle();
                    cellStyle.setFillForegroundColor(HSSFColor.WHITE.index);
                    cell.setCellStyle(cellStyle);
                }
            }

            workbook.write(fileOut);
            fileOut.flush();
            fileOut.close();
            return xlsTempFile;
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

 return null;
    }
}

Now you can open that file with following line if you have any application associated with the xls extension

Desktop.getDesktop().open(xlsFile);

How to show stacktrace, standard output and standard error of each test failure in jenkins mail

Email-ext plugin in jenkins allows customizing jenkins build mail at a programmer's will.


Find you jelly script and go to JUnit TEMPLATE portion of the script.

Here's an example how you can change the script to show stacktrace, standard output and standard error.

 <!-- JUnit TEMPLATE -->  
 <j:set var="junitResultList" value="${it.JUnitTestResult}" />  
 <j:if test="${junitResultList.isEmpty()!=true}">  
  <TABLE width="100%">  
   <TR><TD class="bg1" colspan="2"><B>JUnit Tests</B></TD></TR>  
   <j:forEach var="junitResult" items="${it.JUnitTestResult}">  
    <j:forEach var="packageResult" items="${junitResult.getChildren()}">  
     <TR><TD class="bg2" colspan="2"> Name: ${packageResult.getName()} Failed: ${packageResult.getFailCount()} test(s), Passed: ${packageResult.getPassCount()} test(s), Skipped: ${packageResult.getSkipCount()} test(s), Total: ${packageResult.getPassCount()+packageResult.getFailCount()+packageResult.getSkipCount()} test(s)</TD></TR>  
     <j:forEach var="failed_test" items="${packageResult.getFailedTests()}">  
      <TR bgcolor="white">  
        <TD class="test_failed" colspan="1">  
          <B><li><h2>Failed: ${failed_test.getFullName()} </h2></li></B>  
        </TD>  
      </TR>  
      <TR>  
         <TD class="test_failed_log" colspan="2">  
          <li><h4>Stacktrace</h4></li><li>${failed_test.getErrorStackTrace()} </li>  
          <li><h4>Standard Output</h4></li><li>${failed_test.getStdout()} </li>  
          <li><h4>Standard Error</h4></li><li>${failed_test.getStderr()} </li>  
        </TD>  
      </TR>  
     </j:forEach>  
    </j:forEach>  
   </j:forEach>  
  </TABLE>  
 <BR/>  
 </j:if>  

You can further change the visual effects through css classes.

To know more options of the ${failed_test} object see CaseResult

Where are the Jelly and Groovy template scripts located in jenkins?

The Email-ext plugin page says the templates are located in $JENKINS_HOME_\email-templates_.
I searched a lot to find the html.jelly script as I wanted to make changes to the default.
But didn't find them with any search until I unzipped classes.jar in $JENKINS_HOME/plugins/email-ext/WEB-INF/.

Now to test your changes you have to change the jelly file and jar classes folder again and put in /lib folder.

There is an easier way. Move the classes.jar to WEB-INF folder of email-ext plugin and unzip it. Now you can make changes in the files and test them right away!

You can find a template in my other post How to show stacktrace, standard output and standard error of each test failure in jenkins mail

Create Test Suite for Tycho after login

Sometimes tests fail when they are not actually running. This typically happens in case of integration-tests. When something goes wrong(like login failed) the consequent tests all get failed and the build server starts flooding with the infamous All tests failed! mails!

What to do in such case?
Should we run the tests if the login failed even if they were supposed to run after a successful login?

We faced this recently with our RCP application which connects to a remote server to properly function.
We have around 50 SWTBot ui-tests for testing the features. The tests are run in our CI server.
SWTBot first logs onto the server using a login dialog.
But when the login fails we don't want to run the tests at all, here's what we did:

We created a TestSuite which returns all tests only when login succeeds.
Of course it can be customized further to give more logic to load each test case (like different test cases for different platforms!)

package package.of.test.suite;

import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;

import org.apache.log4j.Logger;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swtbot.eclipse.finder.SWTWorkbenchBot;

/*
 * A test suite which creates tests depending on whether the login succeeded or not.
 * Return all tests if login succeeds
 * Else return a suite containing no tests.
 */
public class AllTests extends TestSuite {

    private static boolean loggedIn = initializeProductForTesting();
    private static Logger  mLogger  = Logger.getLogger(AllTests.class);
    private static SWTWorkbenchBot bot;
    public static boolean initializeProductForTesting() {
        System.out.println("in Beforeclass method before all tests!");
        Utils.login();
        return Utils.isLoggedIn();
    }

    public static TestSuite suite() {
        TestSuite suite = new TestSuite();
        if (loggedIn) {
            suite.addTest(new JUnit4TestAdapter(Test1.class));
            suite.addTest(new JUnit4TestAdapter(Test2.class));
            suite.addTest(new JUnit4TestAdapter(Test3.class));
        }
        else {
            mLogger.debug("Can't log into server!");
            mLogger.debug("No tests will run !");
            Display.getDefault().close();
            System.exit(0);
        }
        return suite;
    }
}

We configure tycho-surefire plugin to run this suite
<configuration>
                    ...
                    <testSuite>bundle.symbolic.name.of.test.plugin</testSuite>
                    <testClass>package.of.test.suite.AllTests</testClass>
                    ...
</configuration>

How to stop massive logging by Birt Chart

While config for birt report gives direct api(EngineConfig) to change logging level birt chart doesn't.
Here's an alternative
            PlatformConfig config = new PlatformConfig( );
            config.setProperty(PluginSettings.PROP_LOGGING_DIR, null);
            config.setProperty(PluginSettings.PROP_LOGGING_LEVEL, Level.SEVERE);

This should stop mass logging by birt chart.

How to cancel an SWT key event?

KeyEvent in SWT has a doit member variable.
Setting it to false cancels any further handling of the event.

public void keyPressed(KeyEvent e) {
              handleKeyPress(e);
              e.doit = false;// cancels the event
}

SWTUtils doesn't support invoking methods with parameter

SWTUtils.invokeMethod allows you to invoke only methods with no parameters.

Here's a customized version of the method that allows you to invokes methods on objects with parameters

    public static Object invokeMethod(final Object object, String methodName,
            final Object[] params) throws NoSuchMethodException,
            IllegalAccessException, InvocationTargetException {
        Class[] paramTypes = getParamTypes(params);
        final Method method = object.getClass().getMethod(methodName,
                paramTypes);
        Widget widget = null;
        final Object result;
        if (object instanceof Widget) {
            widget = (Widget) object;
            result = UIThreadRunnable.syncExec(widget.getDisplay(),
                    new Result<Object>() {
                        @Override
                        public Object run() {
                            try {
                                return method.invoke(object, params);
                            } catch (Exception niceTry) {
                            }
                            return null;
                        }
                    });
        } else {
            result = method.invoke(object, params);
        }

        return result;
    }

Still this method only supports Object[] parameter, it doesn't support any primitive parameter like int, boolean.

VoidResult and its use

asyncExec method of UIThreadRunnable takes a VoidResult parameter.

It's useful for running code in UIThread which involves inter thread communication.

VoidResult interface has a method run().
Put the code in run() you want to execute in UI Thread.
for example you want to call setMaximized(..) method on a Shell you can do this.

Maximize Eclipse active window from SWTBot

Here's a simple way of achieving above
   
private void maximizeActiveWindow() {
        final Shell activeShell = bot.activeShell().widget;
        VoidResult maximizeShell = new VoidResult() {
            @Override
            public void run() {
                    activeShell.setMaximized(true);
            }
        };
        syncExec(maximizeShell);
    }

There's another way of doing above here.

Can't Delete zero byte files in windows?

Here's the solution I found, it works!

Running UI Tests from Jenkins

Usually Jenkins is run as a service while automated UI testing requires active user session.

We faced the problem when we implemented automated UI testing integration with Jenkins.

The solution we applied is installing jenkins.war in tomcat and run tomcat from a user session.

The problem isn't finished there. UI testing requires you always logged in and desktop unlocked.

We need to keep the session alive so that whenever a build starts it can interact with desktop.
We use VDI(Virtual Desktop Infrastructure) in Hyper-V which can keep a session always alive and unlocked.

But when we remotely connect to the session for some maintenance and disconnect again, the session gets locked again.
We used to manually unlock the session from the host server until we found the following solution in stackoverflow .

To keep to session alive after disconnection from Remove Desktop.
Enter following command in cmd prompt.

TSCON {sessionid | sessionname} /DEST:CONSOLE

You can get the sessionid or sessionname with following command

query session

 Example: TSCON 1 /DEST:CONSOLE

Surefire doesn't run the product/application specified

The product or application specified in configuration section of surefire are most probably on another bundle.

1. Add the bundle as dependency to the plugin (in MANIFEST.MF) for which you have written the surefire configuration.
2. Also add org.eclipse.equinox.ds if you have declared the product/application through extension points.

Now your test plugin should be able to access the application/product.

In case you have the extension points and tests in the same plugin/bundle you still have to do 2nd step.

Arguments needed to VM and program for running SWTBot tests with tycho surefire

When you use SWTBot widget finder to get widget with label or title, language comes into the game!
If the application is run with another language in another environment it's bound to fail.

You might think about externalizing/internationalizing your test cases. Is it worth it? ....

Unless you go for i18n you can specify the language as a program argument in surefire configuration

<appArgLine>-ln en</appArgLine> <!--for english-->

As you resolve target platform through tycho you might get out of memory from jvm in projects with enough library dependency.

Specify memory heap in surefire configuration.

<argLine>JVM heap size</argLine>

Specify the arch bootloader constant. why?

Maven doesn't find source in eclipse bundle to compile

By default maven thinks the java packages are in src/main/java/ directory relative to pom.xml location.
But usually eclipse bundle/plugin developers pur packages in src directory.

You can specify the source directory in maven a below
<project>
...
  <build>
    <sourceDirectory>${basedir}/src</sourceDirectory>
  </build>
...
</project>

stitch eclipse + swt + swtbot + junit + maven + tycho + surefire

In jist a high level view of running swtbot ui tests with maven/tycho

1. Specify proper packaging i.e eclipse-test-plugin
2. Specify src directory to maven build, usually it’s different in eclipse than what maven thinks
3. Specify various parameters(memory, language, arch, ws, os, ..) correctly to vm and program
4. Add necessary elements (UIHarness, UIThread…, application…) to tycho-surefire configuration
5. Add all implicit dependencies of application/product as explicit to manifest file of the test plugin. Because tycho cannot resolve implicit dependencies.
6. Add p2 repositories to pom so that target platform can be resolved into pom dependencies.
7. Add Declarative Services library/ dependency to test plugin if it uses any declarative service to initialize application/product

All above steps assume at least mid level knowledge on maven/tycho/swtbot/eclipse/rcp and those who have tried the integration already.

32/64 bit issue with SWT, JVM. Solution maven way, eclipse way

SWT ships different libraries for each Operating System (os), Windowing System (ws) and Architecture (arch).
It doesn't allow loading 32 bit library into 64 bit JVM and vice versa.

So one should specify the osgi.arch value explicitly matching the jvm architecture.

In eclipse you can specify from launch configuration -> Arguments -> VM arguments
-Dosgi.arch=x86  for 32 bit jvm
-Dosgi.arch=x86_64 for 64 bit jvm

In maven there is more portable solution via profile
just activate correct profile on system sun.arch.data.model value

<project>
...
<profiles>
...
          <profile>
            <id>osgi-arch-64</id>
            <activation>
                <property>
                    <name>sun.arch.data.model</name>
                    <value>64</value>
                </property>
            </activation>
            <properties>
                <osgi.arch>x86_64</osgi.arch>
            </properties>
        </profile>
      
        <profile>
            <id>osgi-arch-32</id>
            <activation>
                <property>
                    <name>sun.arch.data.model</name>
                    <value>32</value>
                </property>
            </activation>
            <properties>
                <osgi.arch>x86</osgi.arch>
            </properties>
        </profile>
...
</profiles>
...
</project>

In case you are thinking why not setting -Dosgi.arch value in MAVEN_OPTS or JAVA_OPTS?

These are environment variables and they have to be set everywhere you build your project, all your teammates, build servers...