User tests and change management

As we all know, user tests are one of the best ways to find out about users’ expectations and actual behaviour on a given interface. This is THE preferred method of investigation that we commonly use to qualitatively investigate and report back to our customers the good, the bad, and the ugly of the tested interface. But there is little point in going through users tests if none of the formulated recommendations are considered for redesign. In this article, I want to review how different setups can have an impact on change management depending on the level of customer involvement.

Users’ tests. We love them because we learn so much from them! Yet learning is one thing, actually changing things is another.

For the latter to happen your customer will need to:

  • value the user tests’ exercise as part of the usability consultancy
  • trust the user tests’ findings
  • embrace the recommendations formulated by the researchers
  • act upon them!

Per se, you do not need your customer to get involved in user tests; you can suggest/impose a scenario based on the tasks that you think must be carried out on the interface, test them with users and send a nice report. The problem is that your report is likely to gather dust on a shelf. Because they are a great, user tests can also be very confronting and challenging for customers. It is never nice to discover that a project you spent hundreds of thousands on simply does not deliver. “But if my customer asks for a user test, surely he can expect that, right?” Well… no :) Unless your customer is used to the exercise, he won’t. He’ll just disregard the report of those “only 5” users and move on.

The best way to go about user tests is first of all to sit down with your customer to define the scenarios. Often, you will have to get them to re-focus on the main tasks. They’ll sometimes think the suggested tasks are too obvious to even test and will prefer to focus to be on the new, minor features hidden away in a corner that only 5% of the users need. It is part of the job to challenge your customer on it, and find the right balance between testing the main functionality and still giving feedback on new details.

But the most important part is in the setup of the user tests. Ideally you want your customer to be able to listen and see what is going on during the test (tester screen and facial expressions) and you need someone from your team to discuss the findings with him during the session. By doing so, you give him a chance to take part in the findings and the report you will write will not be a surprise to him. But this does not mean that involving your customer is easy. Who has not witnessed users’ tests where the customer observing the user failing tasks declares “you must have picked the wrong profile”. This is a natural response to the challenge. And this is precisely what needs to be managed.

You will find that often the response varies depending on who is sitting in the room.

The project manager may probably be more inclined to see what is not working.

Marketing managers will be looking at the performance and will be upset when the user does not complete the purchase.

Sometimes you will also have the developers, and it can be quite confronting for them to realise that something they built isn’t working. At first, you will hear comments saying that it is “technically impossible” to do it differently, of course. But once the tension eases, there is room to reassure the customer by stating how common the observed problem is and discuss alternatives. This is the crucial moment where you need to demonstrate your added-value. The facts are clear (the user failed to complete a scenario) and you now need to accompany the change that needs to happen. Listen to the comments made as well as the frustrations. Listen to the 101 reasons why they did it that way. Gently re-state the fact that the scenario failed and suggest alternatives. After the third user failing the same test, they’ll want to change.

Once the frustration is handled, everyone around the table will be in a collaborative mood and you are in charge. You now have great allies committed to make changes.

Through observation of the interactions between people, you can also get a glimpse of the relationships between all these stakeholders within the company. Such user tests sometimes are the first (or at least a unique) opportunity for everyone involved to actually talk to each other about the project, spend time together (user tests usually spanning across one day or more) and it can bring some real added-value. Misunderstandings will come to light, each stakeholder’s planning and objectives will be shared — maybe for the first time. This will help pave the way to the changes that need to happen, with everyone on board.

Once the collaboration is ongoing, ideas will start coming from all directions. The danger then becomes jumping into solutions without first thoroughly analysing all the facts and the problems. This is your job to reset the focus when needed.

At the end of the test, recap on the main attention points that arose. They will become sections of your report. Your customer may ask for input on specific areas which most likely means they trust your expertise! The user test report presentation should cover all the problems identified and suggest changes to the design. Some of the topics observed during the test will now be familiar to those present and this will make the discussion more interesting and focused: you don’t have to convince them of what does not work, they have seen it!

Change does not just happen, it needs to be managed. User tests are a great way to get results if you can become a real partner of your customer!