Six things News UK learned about running user research in Agile

Join Claire Dracott, Deputy Head of User Experience at News UK, and Lee Duddell, UX Director at UserZoom, as they reveal the challenges and rewards of running user research within Agile Sprints.

As more organizations adopt Agile, the traditional means of running user research are struggling to keep up. Quarterly or monthly lab sessions just can’t be wedged into two-week design sprints, and even if they could, there would rarely be enough researchers to run them.

Unfortunately, all too often this means a step backward when it comes to user-centered design (UCD) as Agile teams fall back on hunches to drive their design decisions. But it doesn’t have to be this way!

Many digital teams have successfully adapted to embedding testing within design sprints, reducing the risk of launching digital products with a poor UX – like News UK, the media organisation responsible for publishing The Times and The Sun.

Claire Dracott, Deputy Head of User Experience at News UK, touches on the challenges she’s experienced over the last few years…

“I’ve been at News UK for roughly seven years now, and we’ve been through a massive transformation within that time. When I first started, we were very much a design team, not really user-centered, so I’ve spent the last seven years trying to build that up by bringing research in-house, and really getting buy-in across the business. Which I’m sure most of you know, can be quite challenging at times.”

Here we’ll look at a few things Claire and our host, Lee Duddell, UX Director at UserZoom, have learnt about running UX testing in Agile.

The following insight is taken from a full-length webinar held by Claire and Lee, which can be accessed on-demand: How To Run Research In Agile Sprints By Democratizing It Across Teams. In the webinar, you’ll discover much more detail and guidance on the points we touch on here, so please give it a watch.

The UX team structure at News UK 

At News UK, there’s a UX and design team who sit within technology. It’s a team of 22 people, with a broad spectrum of UX designers, researchers and UI designers. Plus every UX designer and UI designer work specifically to a publication, so they’re not jumping around from The Sun to The Times and vice-versa. They’re also paired together, so there’s always one UX designer and one UI designer working really closely on certain projects.

These UX and UI designers sit together with a product owner, a business analyst, and a few engineers, creating a small cross-functional team. And according to Claire, that’s how they’ve managed to bring Agile ‘in-house’, to make rapid experimental work actually come to fruition and take place within the company.

Being ‘Agile-ish’

Lee describes the problems we face when trying to test within an Agile sprint – and much of it comes from people being used to the old ways of working. In another word: Waterfall.

This method of work was pretty straightforward when it came to UX and usability. You start off with a massive spec, you’re told what to build, it would be documented, and then you’d start making the product. And then there would be stages within the development that would be a good opportunity to get people in the room and run some usability tests.

But as Lee comments,

“Waterfall is out. As “beautiful” as that picture was, it’s all about Agile now. According to the Project Management Institute, 71% of organizations report using agile approaches for their projects sometimes, often, or always – so they may not be entirely Agile, but they have run something in an “Agile-ish” way.”

Lee’s experiences this when talking to customers. They often say, “We’re kind of agile,” or, “We’re agile unless it’s something really important.”

Claire has a similar experience, but the main blocker for her team is the sheer size of the company and the amount of stakeholders they work with. Factors that can be detrimental to the Agile methodology, especially when you always have to bring everyone back in a room, and get buy-in again, and shift direction.

It’s why Claire thinks they’ll never be 100% Agile, but they’re getting there.

Agile’s lack of focus on user research

When adopting Agile, Lee finds that people either start to rely upon live A/B tests to fulfil their user research remit or, worst case scenario, people don’t test at all. This is something Claire advocates never to do. Although the team may be working rapidly within Agile, they always need to be testing with users, and that’s embedded within their ways of working.

It also helps that UCD was embedded in News UK’s previous waterfall approach, so it became part of the culture – especially when everyone from various departments got to see the kind of the successes that user research brings, in terms of products, data and increased subscriptions.

There has been a significant move to A/B testing within News UK, but they realise the need to do more testing in natural environments in conjunction with quantitative methods, so they can better understand the ‘why’ of what is happening.

Participant recruitment

One of Claire’s toughest challenges was all around recruiting participants – making sure they had the right people coming in to do the testing, and that they could source them in time.

As you can see from our latest State of UX research, sourcing the right participants is a key issue for 45% of Enterprise companies…

News UK solved their recruitment issues in two ways – by letting UserZoom Platform take care of all the recruitment within their sprints. Then for in-person sessions, they scheduled a set day every single month to run testing – and this never changes. It means they always know that they need to be recruiting for that specific day. All the administrative tasks gets taken care ahead of time at a regular cadence and it ensures there’s always something being tested every single month.

Claire’s team generally works in two-week sprints, and the monthly face-to-face testing runs in conjunction with other methodologies. UserZoom is deployed to turnaround really quick remote studies, whether it’s think-out-loud or static design testing, to optimize customer journeys. They test frequently, while mixing methodologies and being quite flexible with what they’re working on.

Testable assets

Another common barrier for a UX teams working in Agile, is a lack of testable assets. If you’re on day five of a sprint, you won’t have a finished high fidelity prototype to run UX tests, but you can certainly get feedback on whatever visual asset you have.

Claire’s team do this to help inform things that might be coming up in the next sprint. For example, if they know they’re designing a page, but are not too sure on the ordering of information, they’ll run a test on a simplified prototype.

As Lee points out, sometimes the answers are already out there. Maybe another site has already addressed the thing that you’re working on. Which is why testing on competitor sites is a fast and economical tool for working on usability. As Claire recommends, it’s also helpful to look further afield to someone in a completely different industry, as this may help you push the boundaries.

Stakeholder buy-in

Let’s assume that everybody in the world believes that it’s possible to test within a sprint, and some of us know that it’s a good idea because we’re from a research background, – how do we get buy-in for user research?

For Claire, this was a huge part of her role a few years ago. By her own admission, she went full steam ahead and started doing tests without any kind of backing from most people. But things started to snowball. Once they began usability testing with customers, they ‘over-communicated’ the value of what they were learning.

“You run something and you are talking to everyone about it. You’re saying, ‘We’ve just tested with customers, this is what we’re hearing, this is what we’ve done with it as well.’ That’s what people love to see – that something’s improving.”

Claire made sure she regularly sat down with stakeholders and presented back what she was doing, telling them about the methodologies, why she was doing the testing. And then really going into detail about what they were finding, and what that ultimately meant for the business.

People were seeing the successes, and more and more people were coming to these presentations because they were really interested in them. Claire built a network of allies around the business, who were all saying “Oh, we could do that! or “We could test this!”

So despite struggling to find traction at first, Claire found that by sharing her insights with as many people and teams as possible, this helped democratize user research throughout the company and created many advocates for UX testing.