AWTomation Service

Regular readers of the site might notice that I keep almost deliberately tight-lipped about where I work for real.  That’s right dear reader – Although we live in the bunker – it ultimately doesn’t really pay the bills.  Nope, to do that we engage ourselves within various industries.


I’ve always been a little cagey about talking about employers, and I sort of feel that Titanium Bunker is my thing, and I don’t want to muddy the waters between a ‘professional’ career, and the funny little projects I work on at home.

But I’ve recently been working on a project for my employer that I find personally interesting, and thought I’d write about it here.   Now – to clarify : My employer has not asked me to work on this directly, rather this is a project that I am working on because I think that it will ultimately add value to the business, and it’s also an interesting problem to solve.

So here’s that problem.

My Employer has a wide range of technologies that are employed in combination, to provide a system.  Some of these technologies go back as far as the early 70’s – some are more recent.  My work colleagues have been trying to add automated testing into the solution, but the problems arise where the systems connect.  At the boundaries between technology layers, the ability to continue automating the system under test becomes more challenging. Here’s an example of the sort of problem that we are grappling with.

To start the process, a user would log onto a terminal based application.  This was historically a ‘green screen’ terminal, and floating around the office there are still some old-fashioned dumb terminals.  The terminal access has now been replaced with a terminal emulation process, and that terminal emulation has been improved with a GUI layer wrapping the original terminal screens.

The user would work with that terminal until they had most of the data set up.  Then the system then launches a browser, logging them automatically onto a web site, allowing the user to carry on processing.

My colleagues and I work on the web side of this solution.  I’m sure that you can appreciate the difficulties in transmitting context information between a terminal emulator and a website.  We can emulate this process to a certain extent : we have some libraries that can launch the website with a pre-defined context, but the QA team – quite rightly I think – would prefer to see full end to end processing, as this would give a greater deal of confidence that we hadn’t broken anything.

So I’ve been playing with the challenge : How can I automate and inspect a Java AWT application from a .NET application.  Why .NET?

  • It’s the language we use
  • We already have some automation set up
    • SpecFlow automation testing for the website
    • NUnit automation for unit testing
    • Integration testing through NUnit framework*
  • We have continuous integration running our tests (mostly… well some)
*Purists may gasp here – We decided that NUnit as a test running framework would also allow us to perform integration testing.  We tag all the tests that use ‘real’ resources with the tag “Integration”, and our automated build is configured to ignore integration tests.

First examinations

At first it seemed hopeless.  I’ve used Spy++ to investigate windows and elements before.  Using this technique it could have been possible to look for windows and navigate to elements, and pass appropriate messages using SendMessage – such as BM_CLICK, or sending a WM_KEYDOWN / WM_KEYUP messages, but Java doesn’t seem to play nicely with the Windows OS.  This does raise the question of whether Java Applications can become ‘accessible’.  This is supported by the quickest experiment : A Java Application thrown together using Netbeans reveals that Microsoft Narrator cannot read information about the elements within the application. It can intercept keyboard events, and read the keys as I typed them and was able to see the windows chrome elements but wasn’t able to narrate the contents of the application.

This Stackoverflow article discusses something referred to as the Java Access Bridge which looks like it might provide a suitable solution to this – Here’s a video showing the Java Access Bridge in action.

That seems like a possibility – so let’s put a pin in that.  Instead I wanted to do something a little more direct.  Earlier in this article I mentioned that we have Specflow automation.  We’re using specflow in combination with Selenium WebDriver to control a browser session.  The basic idea is that the webdriver describes a set of functions.  You implement those functions and manipulate your target browser, and Bingo – you can run that same test script across multiple browsers.  I wanted to take that idea of an interface allowing remote control of a device, and instead of just controlling a browser, be able to control an application.


Serving an Application.

So I have been working on what I call the AWTomation library (though that may change) because at the moment I’m automating AWT type applications.  The idea is that a Java application can reference this library and through that act becomes controllable via .NET, or indeed any controller application.  AWTomation is a self hosted web service that executes within the context of the System under test.  While that application is running, it can respond to remote web service calls and perform actions, as if the user themselves were pressing the button.

Currently the service has support for the following functions :

  1. GetForms()
  2. FindWindow()
  3. GetControlsList(WindowName)
  4. GetControlByLabel (WindowName,LabelText)
  5. GetControl (WindowName,ControlName)
  6. ClickButton(WindowName,ControlName)
  7. SetControl(WindowName,ControlName)
  8. TakeScreenshot(WindowName)

This allows me to write .NET consuming code such as :

        [Category("Development Integration")]
        public void TestThatAListOfFormsCanBeRetrievedFromTheSystemUnderTest()
            var forms = rc.GetForms();

Making the application controllable via web services, also has the advantage that the System Under test can be on any machine, meaning that a set of test machines could all run the same set of remote control tests, testing the Java application and launching logic on Windows, Macintosh, or indeed Linux.

It’s early days at the moment – and the project is little more than a proof of concept, however I am keen to try to progress this project forward.  I’ll try and update this blog when I make progress.

Mike is the programmer in the bunker. He writes software for a shaddowy software house in Worcester. He works mainly in .NET and python, but equally has found himself having to support different technologies such as MAXScript and .NET integration. Spending most of his time working on his passion of content packaging, and is currently working on an ebook authoring system for Ubuntu. Although he is the main programmer on the site he doesn’t do much with regard to writitng games

1 comments on “AWTomation Service”

Comments are closed.