12 Steps to Better Code

The ‘Joel Test’ has been around a long time (Joel Spolsky originally wrote the article in 2000), but having only discovered Joel’s blog in the last year, and reading every article back this far, I think this particular post is amazing.

For those not familiar, the ‘Joel Test’ is Joel Spolsky’s 12 simple steps to better code and a better software development process. Lately, I’ve been trying to adhere to these steps in some personal projects, and some development projects I’m involved in at my day job.

Here are the questions to ask yourself when rating the software team you work in:

1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do programmers have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers?
11. Do new candidates write code during their interview?
12. Do you do hallway usability testing?

Read Joel’s full article here, I think it’s great!

LINQ to XML – What I’ve been missing

Ok. You may laugh. I’ve just today used LINQ for the first time to parse an XML file, and I’m seriously blown away at how easy it was. I’m a little embarrassed, since LINQ has been available since .NET 3.5 was released around November 2007.

If you are like I was, (LINQ-less!!), I’ll give a brief introduction here. LINQ (Language INtegrated Query), is a component that adds native data querying capabilities to .NET languages. It can be used to read, parse and write XML files (and also SQL, which I may cover in a future post). Take a look at the example below to see how easy it is to use this technique to read data from an XML file.

Reading data from an XML file is a very common scenario. I always used .ini file as configuration files for any applications I wrote, but .NET doesn’t provide any built in support for .ini, and hence wants you to use XML.

Consider the following XML file:

In order to read this using LINQ to XML, you need to ensure you have specifed the correct header files:


using System.Linq;
using System.Xml.Linq;

Now for the easy part, here’s the code to read data from the XML file, and print out the values to the console:


XDocument xmlDoc = XDocument.Load(@"example.xml");
var servers = from server in xmlDoc.Descendants("server")
select new
{
     Name = server.Element("name").Value,
     IP = server.Element("ip").Value,
     Owner = server.Element("owner").Value,
};


foreach (var server in servers)
{
     Console.WriteLine("Server Name: " + server.Name);
     Console.WriteLine("Server IP: " + server.IP);
     Console.WriteLine("Server Owner: " + server.Owner);
}

Easy huh? The line beginning with ‘var servers=…‘ may look strange to you if you’ve never seen it before, (it did to me). This is an anonymous type declaration. If you’ve never heard of anonymous types in C#, MSDN has some great documentation here.

Happy coding.

Hacker Monthly

A new magazine has just been released. Unlike what the name suggests, ‘Hacker Monthly’ is nothing like Phrack was, in that it’s content is mostly centered around software development, and software topics like Agile development etc.

The first edtion contains an excellent article on why programmers, in the age of dual cores, still struggle coping with parallelism and concurrency when developing software.

Check it out here.

Aside, I see a new issue of Phrack, Phrack 67, is actually being release on July 11th 2010, may be worth a look.

The poor mans OCR

Optical Character Recognition (OCR) has been around a long time. One of it’s main uses, for those not familiar, is to gather text from images.

Off the shelf products, such as Abby’s FineReader exist, with prices ranging from $150 for a single user copy, to up to $10000 for large enterprise ‘site licenses’. But where’s the fun in buying it!

I learnt recently that Microsoft Office 2007 has built in OCR capabilities which can be accessed from C# via a COM interface. I will explain in this post how to leverage these capabilites.

First off, you need to have MS Office 2007 installed. This is obviously a dependency if you develop an application to use the OCR capabilites in the field – it won’t work without Office installed. Furthermore, the OCR capability doesn’t install by default when you install Office, you need to add a component called ‘Microsoft Office Document Imaging’ (MODI).

For instructions on how to add the required component, look here.

Now that you have MODI installed, you can create an OCR application! Boot up Visual Studio and create a new C# console application.

You’ll first need to add a reference to MODI, so we can use it from your application. From the Visual Studio Solution Explorer window, right-click on the ‘References’ folder. When the dialog box appears, select the ‘COM’ tab. Finally, select the object named ‘Microsoft Office Document Imaging 12.0 Type Library’.

The code below will create a new MODI document, retrieve the text, and ouput it word by word to the console, (you could also output it to a text file or a custom XML file, I leave that as an exercise for the reader). I’ve assumed below the image file you wish to retrieve the text from is located at ‘C:\Images’.


// Grab the text from an image
MODI.Document md = new MODI.Document();
md.Create(@"C:\Images\Image.tif");
md.OCR(MODI.MiLANGUAGES.miLANG_ENGLISH, true, true);

// Retrieve the text gathered from the image
MODI.Image image = (MODI.Image)md.Images[0];
MODI.Layout layout = image.Layout;

// Loop through the list of words
for (int j = 0; j < layout.Words.Count; j++) { MODI.Word word = (MODI.Word)layout.Words[j]; Console.WriteLine(word.Text); } md.Close(false);

Notice the 'MODI.MiLANGUAGES.miLANG_ENGLISH' parameter above, this is set to the language you are dealing with. It looks like 22 languages are supported, including Japanese and the Chinese variants.

When I ran this, (using the home page of my McAfee 'Total Protection' 2010 suite as the guinea pig), the results were surprisingly accurate (for English anyway), with only two recognition errors.

Take a look here.

I wonder if it is also as accurate for double byte languages like Japanese etc. I'd also like to check it against an RTL language like Arabic.

Anyway, it's definetely worth a look if you want to develop a custom OCR application on a shoestring budget.

The ‘C’ books are out again…

Three times in the last two years I’ve attempted to become good at programming in the C/C++ programming languages, and three times I’ve either stopped half way through a book/online course, or failed to even start with any momentum.

But, I’ve dug out the books again and am determined to succeed this time. Having come from a Java background, then moving to working with some projects in C#, it seems the natural progression. I’m starting with ‘C in 21 days’, which I estimate should take about two weeks, (as I think I can ignore the ‘What is a variable/array/string?’ sections…). I also just purchased ‘Accelerated C++’ by Andrew Koenig, which I intend to read next. Maybe I’ll post a review here later.

Nothing annoys me more than looking at C or C++ code and not understanding what’s going on…

Test Automation Success Criteria

I’ve been thinking recently about how to validate the success of an automation project, and actually prove a return on investment to anyone interested. This is important for a number of reasons.

Firstly, it shows the QA team, (who are most likely set in their ways with the traditional manual testing approach), that at least some of their work can be completed in a more efficient manner, and testing coverage can be increased without any extra effort on their part. Secondly, it shows management that automation is worth investing in. It also boosts the morale of the people who actually worked on the development of the automation, and gives them confidence to continue and make even more improvements.

Brett Pettichord defines what I think are 4 excellent items to validate any test automation project against:

  1. The automation runs
  2. The automation does real testing
  3. The automation finds defects
  4. The automation saves time

I believe the fourth item above is probably the most important, (assuming 1 and 2 are satisifed). The whole point of test automation is process improvement, and having the ability to absorb more work without requiring additional resources. Item 3 is important also, any defects should obviously be flagged, if not logged automatically also.

If the four points mentioned are satisfied once your automation project is complete, I believe it can be qualified as a success.

Expanding a VM’s Hard Drive

Today, I needed to expand the hard drive in a VMware Workstation 6.5 image. It’s a simple process, but can be non-trivial if you don’t know where to start. I’ve posted the steps below, (much more for my reference than anyone else’s 😉 )

  1. Open a command prompt, and CD to the directory which contains the VMDK file you wish to expand.
  2. Use the command “C:\Program Files\VMware\VMware Workstation\vmware-vdiskmanager.exe –x 6GB ‘Windows XP Professional’”, where ‘Windows XP Professional’ is the name of the Virtual Machine Disk File. This will increase the disk allocation to 6GB, (this may take a while).
  3. Now to join the unallocated space to the primary partition – Ensure your image is powered off, and is set to mount the gparted ISO, (can be downloaded here). This can be changed in the image options under Settings -> CD/DVD. Select the ‘Use ISO image:’ option and point it to the gparted ISO.
  4. Next Power on the Virtual Machine, press ESC at the BIOS screen to get to the Boot Menu. Select CD-ROM as the Boot device.
  5. Gnome Partition Editor will load, press Enter at the boot screen. Select all options as they are asked as default, (ensure you select English as your language.)
  6. To complete the join of the unallocated space to your partition, complete the following steps:

• Once the partition editor loads, click on /dev/sda1 in the partition list.
• Click the Resize/Move button.
• Click and drag the arrow to extend the size of the partition, make sure you do a resize (double arrow) and not a move (four way arrow) so you should have 0mb free space preceding and following, and then click the Resize/Move button.
• Next click the Apply button and then the operation will start, you can expand Details to see the progress, once completed click the Close button.
• Click the power button in the top left corner, then select reboot.
• Edit the VM and remove the ISO from the CD/ROM device.
• When the OS restarts it will do a Check Disk, let this complete, Windows will prompt for a reboot after you login.
• Reboot and load Disk Management (diskpart.exe -> Show volume) and your Primary Partition will be the new size without any unallocated space.

See, it’s as easy as that 🙂 …

Programmatically verify resources in a DLL

I had a requirement recently to be able to programmatically check certain resources were contained in a set of (native) DLL resource files. The idea behind this was to add some post-build automated engineering checks to our existing automated test suite, e.g. ensuring resources for all the required languages have been injected correctly.

I wanted to write a simple C# application to perform these checks. I came accross this handy library which contained functions for almost all the functionality I required:

ResourcesLib

Using this, we can perform functions such as importing resources, loading strings, and even injecting new resources.

For example, here’s how you could retrieve all languages contained in a resource DLL:

string file = "resource.dll";
RawResourceFile resFile = new RawResourceFile();
resFile.Load(file);

for (int i = 0; i < resFile.Languages.Count; i++) { Console.WriteLine("Language: " + resFile.Languages[i]); }

Download the library and take a look, it can save you lots of time if you need to perform any checks/actions on DLL file.

Developing Language Independent test automation

At my day job, we’re currently developing an automation suite to perform Build Verification Testing (BVT) and some basic functional testing on new builds coming into QA. This is a challenge in itself, but becomes even more difficult when you consider we currently release products across 28 distinct locales, including right-to-left languages like Arabic and Hebrew.

The drawback of this, is that we need to think of the bigger picture when writing an automation script/program. A program we develop to perform some action on an English build, on an English XP, may not perform as expected on say a German build on a German language version of XP.

Here are some general guidelines I’ve learnt along the way, to keep in mind when developing automation scripts/programs that you intend to run accross multiple languages/platforms.

1. Never hard code language dependent information

I’ve seen this a lot. Such information may be the expected title of an alert, that appears when the script performs some action. Hard-coding the expected alert title in English will cause the script to be useless on any other language. All these strings should be externalised in some way, even to a simple text file.

2. Never hard code paths to Windows system folders

Hard-coding these paths, such as the paths to ‘Program Files’ or the ‘Documents and Settings’ folder will cause your automation to fail on non-English platforms. The problem with these is that they may be localized on some environments. For example, ‘Program Files’ becomes ‘Programme’ on a German environment. If you need to use these paths, always use the Windows environment variables to retrieve the value, that way you can be sure the path will be valid for that platform.

For example, retrieving the path to the ‘Program Files’ directory (in VBScript):


Set oShell = CreateObject( "WScript.Shell")
strProgramFilesDir = oShell.ExpandEnvironmentStrings("%PROGRAMFILES%")

Where an environment variable is not available, you can usually find the path in the Windows registry with some Googling. For example, the ‘All Users\Application Data’ folder can be found at the registry location below on all Windows variants:


HKLM\Software\Microsoft\Windows\Explorer\Shell Folders\Common AppData

3. Don’t assume elements will always be in the same place

In controls such as drop-down menus, the order of the items will be different on each language. Assuming the same order will cause unexpected results or cause your automated tests to fail outright. Also, in RTL languages such as Arabic, the elements themselves will be in a completely different area on the dialog.

4. Never hard-code date formats

Always use the date and time formatting functions provided by the development language you are working with. Also, a common error I’ve seen is that developers assume the date seperator used is always the same, not true! This is a ‘-‘ (hyphen) on an English platform, whereas a ‘.’ (period) on a German environment.

For example,


31.12.2009 - Germany
31/12/2009 - Belgium
31-12-2009 - Ireland

5. Never use ‘record and playback’ automation tools

These are pretty useless in an environment where you are attempting to develop automation to run across multiple languages, with the obvious example being attempting to run a script on an Arabic build which you previously recorded on an English build, where the dialogs are mirrored and the elements are in completely different areas of the screen.

6. Create automation which is resistant to changes in the UI

A couple of string changes in the UI should not cause your automation to fail. Use Object ID’s where possible, as these will rarely change, and it’s easy to add a fix if they do.

Just adhering to the above simple guidelines should solve many of the common issues encountered when attempting to develop an automation suite to run across multiple languages.

Automating Virtual Machine operations on ESXi Server from C#

VMware provides two really useful API’s for automating virtual machine (VM) tasks on both VMware Workstation and VMware ESXi server.

  • VI Infrastructure API
  • VIX API

These are extremely easy to use from C#. In a QA environment, the automation of VM’s can be hugely benifical, wheather attempting to automate an environment for build sanity checks or functional tests.

This post will outline the basics of using the VIX API from C#, in order to perform operations on VMware ESXi server. If you don’t have access to an ESXi server, you can install it on a VM, it’s free to download from the VMware website!

For starters, you will need to install the API’s on your development machine. In order to download, you will need to create a VMware account, which you may already have if you have downloaded Workstation or ESXi server in the past. If you dont, you can create an account for free. Once logged into your account, you can download both API’s from the ‘Support & Downloads’ section.

Let me explain the differance between these two API’s. From VMware’s own documentation:

The VI API provides access to the VMware Infrastructure management components—the managed objects that can be used to manage, monitor, and control life-cycle operations of virtual machines and other VMware infrastructure components (datacenters, datastores, networks, and so on).”

VIX on the other hand, is used to automate the actual operations on VM’s, such as booting them, copying in files, getting/setting VM environment varibles and other tasks you may wish to perform. The coolest part of VIX is that a wrapper for C# exists, created by Daniel Doubrovkine over at dblock.org. This wrapper, ‘VMwareTasks’, provides a simple object-orientated approach to VIX, which will be familar to C# developers. Download the wrapper here.

Now for the basics of using the VIX API and VMwareTasks wrapper. Create a new console application project in Visual Studio. You will need to add a reference to the VMwareTasks DLL, which is located in the ‘bin’ directory when you extract the VMwareTasks download.

Look how simple it is to power on a VM!


// Declare a new virtual host
VMWareVirtualHost host = new VMWareVirtualHost();

// Connect to the ESXi server
host.ConnectToVMWareVIServer("192.168.1.39", "root", "password123");

// Power on an existing VM by name
VMWareVirtualMachine machine = host.Open("[datastore1] XPP_SP2.vmx");
machine.PowerOn();

The simple code above just connects to an ESXi server, and powers on an existing VM, but you can see how easy it is to perform operations on VM’s.

Here’s how to create and revert to a snapshot:


VMWareVirtualHost host = new VMWareVirtualHost();
host.ConnectToVMWareVIServer("192.168.1.39", "root", "password123");
VMWareVirtualMachine machine = host.Open("[datastore1] Vista_EN.vmx");
machine.PowerOn();
machine.Login("Tester", "testing");

string snapShotName = "base";
machine.Snapshots.CreateSnapshot(snapShotName, "Clean");
machine.PowerOff();

VMWareSnapshot snapshot = machine.Snapshots.GetNamedSnapshot("base");
snapshot.RevertToSnapshot();

Or to create a directory:


machine.CreateDirectoryInGuest(@"C:\TestDir");

You can see from the above examples how easy it is to perform operations on VM’s using these API’s. Install the API’s and play around with the functionality, I guarantee you’ll be impressed!

Blog goes live!

Welcome to my blog! Live as of January 31st 2010 at http://www.jimmycollins.org/blog.

I’ve just installed WordPress in order to get this blog up and running, and I have to say it was one of the most flawless processes I have ever been through. The famous WordPress ‘5 minute install’ actually took me around 3 minutes.

I had intended to develop a custom blogging application as an exercise in C# or PHP, but really didn’t have the time. Maybe I will do this in the future, but for now I’m pretty happy with the WordPress installation.

I intend the content of this blog to be mostly technical. Likely topics include C#, PHP, VBScript development, QTP automation, and Virtualization topics such as VMware ESX server management.

More soon!