I think people have a tendency to greatly underestimate the difficulty of writing a good GUI-automation suite. It’s not as simple as record and playback, and it’s not like building regular software.
I’ve seen experienced developers underestimate it many times. Inevitably, they end up getting very frustrated and complaining about how rubbish the automation tools are. And yes, the poor quality of automation tools is definitely part of the problem. Developers are used to using very polished tools, lovingly crafted by developers, for developers. Testers are used to using either bloated overpriced commercial tools, condescendingly oversimplified so that “anyone” can use them, or well-meaning but under-maintained free tools, struggling to keep up with the latest technologies.
Let’s face it, the choice of tools isn’t great
Every time a new browser version or web technology hits the market, GUI automation tools suffer for it. Did any other test automators get a slight chill when they heard about HTML5? We only just recovered from the introduction of AJAX.
To be honest, I don’t think cross-browser GUI automation provides that much value anyway. Most cross-browser bugs are not the kinds of bugs that are likely to be detected by automation. The amount of effort that goes into to trying to get the automated tests to run cross-browser and cross-platform is usually not worth the amount of bugs found. If automated test tools were better, it would be a simple matter of changing a setting to make them all just work. But to my knowledge and experience, most if not all of them are just not there yet.
Experienced test automators write a framework on top of the out-of-the-box automation framework for all of their automation projects. Some people keep a generic copy of this framework that they can take from project to project. Why is this even necessary? Why isn’t it just built into the existing framework? Sure, everyone has their own special implementation, but the underlying principles are usually the same.
It’s still evolving
Then things like Project Sikuli come along. It uses image comparison to find controls in the app. As in, you can see pictures of buttons and stuff in the automation code. When I first heard about this, I thought it must be dodgy as all hell. Automated tests that use bitmap comparison to verify images have always been very unreliable in my experience. But I have been reliably informed that this does actually work. Is it a good idea? Honestly I have no idea. I have a hard time working out how this would be advantageous. But it is interesting because it is a more true version of GUI automation. It’s not just looking at the DOM anymore, it is actually looking at the GUI. This could make it about as close to the actual user experience as any automated tool has managed yet.
As frustrating as it can be, GUI-level automated testing is still pretty new and still has some growing up to do. The tools are improving as GUI-level automation becomes more popular. It will be interesting to see what direction it takes.