24. Jul 2015
XITASO Tech Talk: Coded UI Tests with .NET
Benjamin Winter shared his experience in UI testing with his colleagues at XITASO in one of the weekly tech talks.
This tech talk was an interactive session where we recorded some UI tests with the Coded UI Test Framework (CUIT), which is integrated in Visual Studio 2013 Premium (or above). The first test recorded some clicks on a MVVM-calculator. Then we added some assertions to ensure the calculation we made was correct. We experimented around with the AutomationId and IsEnabled-Properties to realize how easy it is to accidentally break a UI test. After the tests within the sample project, we recorded some UI tests on internally developed software. This software contained some more complex controls. To add assertions to these controls, it was necessary to use advanced navigation techniques inside the Add Assertion-Window. Finally we explored some of the capabilities of CUIT to record any non-WPF controls by recording a click sequence inside the Eclipse development environment. Some of the actions were replayable while others failed.
Benjamin got his Master of Science (Informatik) from the University of Augsburg. After finishing his studies he directly started his professional career as software developer at XITASO in 2014.
Benjamin Winter, Software Developer
benjamin.winter@xitaso.com
What is the main advantage of coded UI tests in general?
Benjamin: Coded UI tests can test exactly what a user will see. They can be faster than manually working through a test specification and doing all the actions by hand. Especially to prevent regression bugs it can be useful to create a UI test and then fix the original bug.
You presented how to record UI tests in VisualStudio. Is this way thus limited to projects written in Microsoft related languages?
Benjamin: No. The tests can record any action on a Windows desktop. But there may be some difficulties when replaying the test, which will need further tweaking in the recorded steps. For example, in our test with the Eclipse actions, the “Run” button could not be clicked by the test framework. It always complained about the button being overlapped by another control.
Why are UI tests so expensive?
Benjamin: Coded UI tests are so expensive because it is more than just recording clicks. You should also take care that the clicked items have proper names in the generated code. So, if a test fails, you know what went wrong. Additionally, when you make assertions you have to take care of localization of the buttons and somehow wire them to your localization logic. The assertions you can add via the test recorder are limited to string comparisons. Thus, if you for example want to ensure that a control value is larger than a reference, you would need to modify your assertion manually. This makes clear that coding UI tests is more than just recording clicks. Good coding skills and expertise of the used framework (and how it works) are necessary. Coded UI tests require at least as much work as implementing a whole User Story.
How about the others? Did you learn anything new from today’s session?
Andreas Stuiber (Team Leader, at XITASO since 2012): I got some new ideas how coded UI tests could be used to simplify parts of our system tests that are executed in every sprint to make sure that the system doesn’t regress.
Christian Mika (Scrum Master, at XITASO since 2015): Just to make it more specific: I would really like to integrate automated UI tests in our testing strategy as far as possible. As automated UI tests are difficult to maintain and costly this could be an interesting challenge. However, in my opinion, UI tests alone won’t be sufficient as system tests. You need all the parts of the testing pyramid Benjamin showed during his talk.
Take home message:
- Coded UI tests are powerful but complicated to create/record.
- Coded UI tests should be the smallest part of testing an application. Nevertheless it probably takes as much effort to create them as creating the other tests (e.g. unit tests, integration tests).
Further reading: