How to test a new website or concept

When developing a new site or service proposition, it’s essential you get feedback from your target audience.

There is a distinct case for running usability testing on projects, however when it comes to design research, things are a lot less clear. This can make it tough explaining to clients or even colleagues the value of gathering research when a project has not yet been defined.

Clients can see the value of getting feedback on how to improve something that already exists, but it’s a lot harder to convince them that they need to listen to people who have not seen a product that doesn’t exist yet (especially when stakeholders already have ideas about what is needed).

However, gathering feedback from target users early on can generate feature lists and map out the scope of a project, making it more likely to meet user needs in the long term. You can also save time and money by avoiding costly mistakes, and get ideas and inspiration all-in-one.

So how can you do this?

Firstly, you have to be clear with clients about what they can expect from testing concepts with target users. Failing to set expectations will result in stakeholders being unlikely to understand what you have learnt. I think it’s common for stakeholders to believe they know everything about their target audience, however this is rarely the case even if they do know a lot about their demographics. Most stakeholders have idealised views about how people want to interact with their company, which can lead them to make assumptions.

It’s easy to fall into the trap of believing that there is no point talking to potential users as they wont want a new product or service (I’m thinking of that infamous, highly debated Henry Ford quote here*). To an extent this is right, people can’t articulate things they don’t yet know of, however they can articulate in great detail the problems they currently face when completing tasks. This information is pure gold: it’s what will give you the ideas for new features and approaches that will really make your audience happy. Ignore this at your peril.

You can gather some great data from running ethnographic-inspired research in the field, however this is rarely feasible in agency or commercial environments, so my collection of three techniques focusses on more practical ways you can gather research.

Three approaches for testing concepts

There are three main approaches I have used when running this type of research. Each involves the combined use of depth interviews and observation to supplement the technique and provide background information. They each have different pros and cons, which I am going to share with you.

Comics

Making comics is a low cost and quick way of mapping an experience in an easy to communicate way. They are a good way of communicating both the on, and offline experiences a user can have with a site and service without the need to develop ideas beyond concepts.

When I use comics like this, I keep the illustrations simple, and the visible interface minimal and to the point. You don’t need to go in to minute detail – you are telling a story not describing every last interaction.

Illustrating scenarios

The comic is a great tool for sharing scenarios using storytelling devices. By incorporating this storytelling aspect into the research, you can build a rapport with participants asking questions like “have you ever done that?”, “how would you have done that differently?” etc, without getting bogged down in language that might not even exist yet (like if the service is really new).

When you are picking scenarios to illustrate, keep them to the point – your comic should only be one or two pages long (between four and eight illustrations per page). If it’s getting longer then you are trying to explain too much, and should probably break it down into smaller stories. Your participants (and stakeholders) will struggle to follow longer stories and you will also find it harder to use the comics to prompt engagement from them.

I create my comics using Adobe Illustrator and the wonderful Comic Life by Plasq.

Pros:

• Cheap
• Quick
• Helps create rapport with participants

Cons:

• Unsuitable for some subject matter
• You need some drawing and storytelling skills

Whiteboard sketching

It can be tempting to ask participants to draw how they do activities or brainstorm ideas during depth interviews. The problem is, most people are scared of drawing or even writing something under the scrutiny of another person, let alone a stranger. They are unlikely to be able to let themselves go enough to share what they have inside them. Pen and paper is just too permanent for most people.

This is where magnetic whiteboards are great. You can buy printer paper that is magnetic: it runs through most office printers. This makes it possible to create custom designed magnets really quickly. I have made magnets of people and digital devices to use as prompts for participants to use to re-tell and imagine scenarios on whiteboards. I give them erasable pens too, so they can add additional items or draw arrows between objects or processes. If you start the process off with a warm up activity to get participants feeling comfortable drawing and erasing, it takes the pressure off them and they can complete the tasks much more comfortably.

This gives participants the tools to visually articulate how they imagine using systems or services without the constrictions drawing on paper would give them. It also frees them up to walk about while they are doing the activity which seems to relax them more than being hunched over a table with some felt tips.

When participants have finished drawing and assembling the magnets, it’s really easy to take a photograph and get them to do something else.

Pros:

• Cheap
• Quick
• Great for multi-platform services

Cons:

• Unsuitable for services that are only on one platform (e.g: mobile)
• You need to run the interviews where there is a big magnetic whiteboard
• Some participants will struggle to get over feeling self-conscious

Hand-drawn wireframes

Really rough hand sketched wireframes can be used as a great conversation starter with participants. The low-fi look and feel makes them friendly and easy to talk about – there is no fear of offending the moderator with negative comments as the sketches are clearly just work in progress.

By illustrating the basic functionality and flow of the site, you can simulate the planned experience just by asking participants what they would do next and what they expect to happen. Asking them to reference other sites will help you to build up a picture of the interaction behaviour they are expecting from the proposed sketches.

You do need to be able to draw and write relatively well, and consistently to do this. If its not your strong point, you can always draw the wireframe using a program like Keynote, Powerpoint or similar and print it, then trace that.

Pros:

• Great if you have already done some planning and want feedback on messaging as well as flow

Cons:

• Takes longer than the other two methods to generate materials to test with
• Not all participants will engage well with sketched wireframes
• Some stakeholders may find it too prescriptive

A note about content

It’s worth noting that in each of these examples you can learn a lot about what users need from content on the site, language and messaging. It’s best to include sample content in all of these scenarios, as if you only include dummy lorem ipsum you are not going to get any real feedback on tone of voice or positioning. You do not need to be a copywriter to be able to include some example copy, and it really does reap rewards: it will help significantly later in the project.

*Here is that Henry Ford quote, re-quoted by Steve Jobs for extra wallop on the CNN blog.