Why Designers Are Usually Bad At Designing Tests (& How To Avoid This Trap)

Fellow designers! Those of us that spend a lot of time designing websites, experiences, apps, books, cars, brands, labels. We’re bad at designing user tests. We’re bad at it because most of us have never done it before. Many clients accept work that is untested. Freelance designers, have you ever designed something and then submitted it to a client for review without testing it out with a fellow human (that is not your studio mate)? Sure, a client may disagree with your choice of colors, fonts or branding, but compared to USERS, clients are relatively easy to convince. Why? Clients are on a deadline with a limited budget, and they will accept design work that is untested by users because they want nice things quickly. Many of us design right up until the deadline without allotting a few hours or days to testing with users. Don’t skip the testing period. Before you submit it, take a minute to design a test for the product or prototype that you’ve just designed. Again, designers are usually bad at this. We don’t design tests. We’re too busy designing websites, presentations, products. Important things. But Scientists do it all the time. In fact, they don’t make anything without thinking about how to test and measure it. I like to think of scientists as designers who design tests for a living. If possible, think about testing while you design the product. It will really help your creative process, rather than hinder it. Before the test Assuming you have a target user, find a group of people that you want to design this test for. Start asking people now for their time. People are the hardest and most unpredictable part of this entire process. Don‘t waste their time. Think carefully about what you want to learn from the test. List 15 things from simple to hard. Then rank your top 3. List 5 assumptions. What are you assuming about your design? About your user? If possible, print out the website, product, or whatever it is you are testing. Print out each and every stage. For each “stage,” write down a question on a post-it note. (Does the user understand what this button does? Is the CTA clear?) Seeing things in physical form in front of me helps me edit it down and find errors I didn’t notice earlier. Then build the test. Build just enough to learn. Do not design endlessly. Design for the test. Be ruthless in your efficiency and focus. If the test is in person, it may be as simple as showing someone a few pages of a website. It’s very different if it is conducted remotely! Do not underestimate how much harder it is to collect data if doing so remotely. Plan for how you will collect data now. What quantitative or qualitative things are you hoping for from your users? How will you measure success? During the test Get some back up. No one can record and engage the user at the same time. Record the process so you can share it with other people. Pretend they will not believe you unless they see it for themselves. Video is great. Audio at a minimum. Measure it. Think creatively about how to do this. Signs of engagement can be super simple. But you’re probably not measuring if you don’t use units somehow. Minutes, dollars, seconds, votes, # of sighs, # of laughs? Which page does the user dwell on the longest (seconds)? Give them a fake budget at the beginning — where do they spend their dollars? Do they let out an audible sigh of delight? How many? Maybe they click on things they like the best and ‘vote’ for you? Collect and capture this data. After the test Aggregate all your findings but do not judge them yet. Gather each idea on 1 post-it note and just spend some time pulling it together. Then prioritize feedback. Were your hypotheses proven true? What feedback is actionable now, what is not actionable now? Consider the “order of operations” when incorporating feedback. Deal with game-changing feedback first, not opinions on color schemes or fonts. Prepare a quick Keynote to share your results with the team, even if you don’t have an audience. This will force you to synthesize your results. Timebox it and give yourself just 20 minutes. (No messing around with graphic design stuff!) This should be do-able within 1–3 days depending on the size of the test and how many testers you have lined up. Rinse and repeat as necessary.