Automate your Android App Testing Using Robotium

I like that it’s the time of the year when I actually have some time to invest in blogging here. Updates have been pretty sporadic this year (another item to add to my list of resolutions once 2013 hits in a few days).

In this post, I’d like to talk a little about Robotium, and how you can use it to automate the QA of your Android applications. Mobile applications are perfect for Quality Assurance automation – they are usually small, have limited functionality when compared with desktop applications, and are rarely complex in terms of testing steps. Also, if your a solo mobile developer, you may not have the time (or money) to invest in proper quality assurance of your applications.

We’ve been using a Robotium based solution written in Java at my day job for a while, to validate the quality of our localized product builds, both as a unit testing tool and as an actual test automation tool used by our QA team.

One of the main reasons why we chose to invest time in Robotium is the fact that it offers flexibility in terms of how it recognizes objects within your app (e.g. like it would have to in order to click a button). This is especially important for us, as we want to develop a solution once, and have it support multiple language versions of our application. We also looked at MonkeyRunner, which ships with the Android SDK. MonkeyRunner uses Jython (a Python implementation in Java) scripts to walk through applications.

I prefer Robotium as MonkeyRunner lacks the tight UI integration offered by Robotium, (although MonkeyRunner is much easier to setup and run).

When developing automated test scripts using Robotium, we can use the text from a UI object to identify that object at run-time, for example the label on a button. What we do is compile the localized resources from all our languages with our test application, so if we switch our device language to German for example, the resources from the ‘values-de’ folder are loaded by our test application, and thus we can add new languages to our automation solution simply by re-packaging the APK file containing our test automation scripts, and it will run on any of our supported locales without code modifications.

One of the disadvantages of this approach, (and almost all UI based test automation solutions to be fair), is that the maintenance may be high – for example, you will probably lose 90% of your code in a UI refresh.

Creating a Simple Robotium Automated Test Case

  • First off, download the latest version of Robotium, (3.6 at the time of writing).
  • Now simply create a new Android application in the IDE of your choice, I prefer NetBeans due to familiarity, but I know most people like Eclipse for Android development.
  • Add the Robotium JAR as a reference in your project.

For the purpose of this example, let’s say we want to click the ‘Cancel’ button in the below screen capture. As I mentioned above, our Robotium code uses the string ID’s to find objects (based on the current device language, we will look for the value of that string ID for that language if available, and use that to look for the object on the current screen).

MMS Image

Code (Finally!)


package com.jc.robotium

import android.app.Activity;
import android.test.ActivityInstrumentationTestCase2;
import com.jayway.android.robotium.solo.Solo;
import android.content.res.Resources;
import android.content.Context;
import java.util.Locale;

public class RobotiumExample extends ActivityInstrumentationTestCase2 // Provides functional testing of an activity
{
    private static final String TARGET_PACKAGE_ID = "com.yourcompany.yourapp";
    private static final String LAUNCHER_ACTIVITY_FULL_CLASSNAME = "com.yourcompany.yourapp.Activities.MainActivity";
    private static Class launcherActivityClass;
    
    private Solo solo;
    private Activity activity;
    private Resources res;
    private Context context;
   
    // This will launch the application specified above on the device
    static 
    {
        try
        {
            launcherActivityClass = Class.forName(LAUNCHER_ACTIVITY_FULL_CLASSNAME);
        }
        catch(ClassNotFoundException e)
        {
            throw new RuntimeException(e);
        }
    }
    
    public RobotiumExample() throws ClassNotFoundException
    {
        super(TARGET_PACKAGE_ID, launcherActivityClass);
    }
    
    @Override
    protected void setUp() throws Exception
    {
        activity = getActivity();
        solo = new Solo(getInstrumentation(), activity);
        context = getInstrumentation().getContext();
        res = context.getResources();
    }
    
    public void testUI() throws Throwable
    {
        // ....
		
	// Robotium code begins here
		
	// Wait for the 'Cancel' button
	solo.waitForText(res.getString(R.string.cancel));  
		
	// We may want to take a screen capture...
	solo.takeScreenshot();
		
	// Click the cancel button
	solo.clickOnText(res.getString(R.string.cancel)); 
	
	// ....

    }
    
    @Override
    protected void tearDown() throws Exception
    {
        try
        {
            solo.finalize();
        }
        catch(Throwable e)
        {
            // Catch this
        }
        getActivity().finish();
        super.tearDown();
    }

Notice a few things above:

  • We are extending the ActivityInstrumentationTestCase2 class, which provides us with the ability to run test methods on the UI thread.
  • The method that contains your test logic must be prefixed with ‘test’, notice mine is called testUI().
  • We could have looked for the ‘Cancel’ button by passing the text ‘Cancel’ to the waitForText() and clickOnText() methods, but that would only work on the English version of our product. Instead we use the string ID, and look up the value of that string ID based on the current device locale.
  • If you use the ‘takeScreenshot()’ method I have shown an example of above, screen captures will be saved in ‘/sdcard/Robotium-Screenshots/’ on your test device.

Running the Test

Compile the APK containing your test automation script and install it on your test device, along with the APK under test. One ‘gotcha’ here with Robotium is that the APK under test and the APK containing the test logic must be signed with the same certificate. This may not be the case if your APK under test comes out of a build system of some sort. You can use resign.jar to re-sign your APK under test with the same signature as your APK containing the test logic (by running it on the same machine on which you compiled the test APK).

Once both are installed on a test device, we can launch the test from a command prompt via adb:

adb shell am instrument -e class com.jc.robotium.RobotiumExample -w com.jc.robotium/android.test.InstrumentationTestRunner

Conclusion

Robotium is a good test framework for getting automation for Android applications up and running very quickly. It has an active community and updates are released regularly, the latest version (3.6 at time of writing), also seems to be a lot more stable than previous versions.

On the flip side, since we are relying on text resources (which may change frequently), the maintenance on Robotium based automation solutions can be high if your UI changes a lot, which is probably a high possibility for a mobile application.

If your looking for an automation framework for your Android application that you can get up and running quickly, and use to run scripts across an Android application supported in many languages, I would recommend Robotium, and will be keeping a close eye on it as it develops further.

Some Useful Android Debug Bridge Commands

As I mentioned in a previous post, the Android Debug Bridge is a hugely useful command line tool which installs with the Android SDK, to the ‘platform-tools’ directory.

If your doing any development or testing on Android devices, this is a must to have installed on your machine. I recommend adding it to your Windows PATH variable, as once you begin to use it, you’ll find yourself coming back to it more and more.

In this post, I’ll outline some of the most useful commands I’ve been using in my interactions with Android so far. First off, ensure you have an Android device connected to your machine, or an emulator running.

adb provides a command for determining what Android devices are detected to your machine, ‘adb devices‘:

adb devices command

Use the above command to ensure that adb detects your device or emulator. For example, it may not detect a physical device if you don’t have the correct USB drivers installed.

Here are some of the other commands that should prove useful to you when working with Android devices. Note that for some of these commands to work, you will need to a rooted Android device.

Install an APK file

adb install path_to_apk_file

Uninstall an APK file

adb uninstall com.myapp.main

View the device log buffers

adb logcat

Switch locale on the device (to German in this example)

adb shell “su -c ‘setprop persist.sys.language de; setprop persist.sys.country de; stop; sleep 5; start’

Start intent e.g. your applications main Activity

adb shell am start -a android.intent.action.MAIN -n com.myapp.main/.activities.MainActivity

Run all Instrumentation tests in a Java class against an application

adb shell instrument -e class com.myapp.tests.UITests -w com.myapp.test/android.test.InstrumentationTestRunner

Get a dump of the devices configuration (useful to include in bug reports)

adb bugreport

These are the commands I’ve found most useful when working with Android so far, view all available commands by typing adb -h from the command line.

Happy New Year.

Back To Java & Some Android Test Automation

I’ve been getting back into some Java programming lately, and for the most part, have enjoyed it immensely. I say ‘for the most part’, because I initially had the misfortune to install the Eclipse IDE. What an absolutely horrible application. OK, it’s great that it’s a free development environment, and for the most part works as expected, but it’s not the most intuitive application to use. One of my main annoyances also was the large amount of Eclipse related project files created for simple projects.

For example, for a simple application created in Eclipse, my ‘.metadata’ folder contain 546 files! I honestly have no idea what 95% of these files were for, and they changed. They changed a lot. For reasons I have no idea of. This really annoyed me as I use SVN as my source control system, and had permanently red folders – I’ve got a touch of OCD when it comes to knowing exactly what files I’ve changed and for what reason. Eclipse lasted three days on my system (two of those were the weekend).

Enter NetBeans 7.0.1. Having last used NetBeans around version 4 (around 2007 I think), I was expecting radical changes. There are some major changes, but to my surprise, all for the better. The simplicity of the IDE has been retained – everything was either where I remembered it, or where I expected it to be – a far superior machine to the aforementioned Eclipse.

My main reason for interacting with Java again, is related to one of my current work projects. I don’t have to say that the mobile applications arena has exploded in the last few years – that statement is quickly becoming a clichĂ©. One of the areas which needs to receive significant attention in my opinion is the area of mobile test automation. Some great tools already exist, but many are in their infancy. The problem also grows when you think of the requirement for any mobile test automation system to work across multiple versions of Android, and also multiple languages.

Doing some research on Friday afternoon, I came across MonkeyRunner. This is an Android supported API that provides the ability to interact with and control Android devices (physical or an emulator).

Deciding to take a look, my first hurdle was setting up my NetBeans IDE to work with the Android SDK. The Android developers documentation is heavily biased towards Eclipse, why I’m not sure. Having been sure someone had faced this challenge before, I turned to Google, and sure enough Binary Wasteland had an excellent article on setting up the Android SDK in NetBeans.

Next question – which test automation requirement to try to implement? The answer – the ability to take screen captures of applications running on an Android device. Our localization department use screen captures heavily, for unit testing of localized UI’s, screen shots for documentation, marketing requests etc. So such a feature would have to be included in any mobile test automation system.

To digress for a moment, I have to mention ADB. The Android Debug Bridge is an excellent command line tool that installs with the Android SDK. It allows you to connect to your Android device (wired or over WiFi), and install applications, remove applications, get certain properties of the device (e.g. the language), dump the entire configuration from the device for use in bug reports, along with many other features. Take a look in the ‘platform-tools’ directory in your Android SDK folder, adb.exe will be located in here. I suggest adding this directory to your Windows PATH variable, as you will find yourself using it a lot.

But back to the feature for taking screen captures of an Android device. This is quite simple. Some examples I found on the internet seem to be out of date since Android 4.0 (Ice Cream Sandwich) was released. There seems to be a new package called ‘ChimpChat’, which contains a wrapper for the ADB, and also the functionality for taking a screen capture previously contained in the MonkeyRunner package.

Time for some code:


import com.android.chimpchat.core.*;
import com.android.chimpchat.adb.*;

public static void main(String[] args) 
{ 
        AdbBackend adb = new AdbBackend();
        IChimpDevice device = adb.waitForConnection();
        
        IChimpImage image =  device.takeSnapshot();
        image.writeToFile("C:\\Test.png", "png");
        
        device.dispose();
}

The above code will do the following:

  • Create a new instance of the ADB wrapper.
  • Create an IChimpDevice, and wait for a connection.
  • Take a screenshot, and save it to the location specified.
  • Close the connection to the device.

I have some questions though. ADB seems to only support one connected device, and always connects to the first one it finds if more than one device is connected. What if I want to interact with a number of Android devices, e.g. a mobile phone and a tablet? I need to look into this.

I also need to figure out how we will interact with our complex mobile application UI’s, how specific UI elements will be identified, and if we can programmatically switch the language settings on attached Android devices.

But the above is a good start towards a re-usable mobile test automation system.