How You Might Compromise Your Design Research

Sketching out wireframes as hypothesis after the first round of design research Validity in your creative research Hi there, this is my first blog post. From now I will publish posts about design. All certain kinds of topics that interest me and want to share my thoughts about. A follow or a like is always appreciated and please feel free to ask about or challenge any statements I make. I study communication design in The Netherlands. Within the curriculum we did a seminar exploring topics of our own choice. I researched mistakes people make when doing research as input for their design projects. My focus of this article is not to make you an academic researcher. What I would like to do is create awareness of your research approach. Your current approach might give you answers that don’t resemble reality. Caring for the validity of your design research allows for three things: It helps in making design decisions, instead of designing based on your personal opinion. You can be more ensured you are building the right product. Caring for valid research allow you to persuade or convince clients, stakeholders and team members about your design decisions. Validity — definition statement As I just stated; this is not about teaching you an academic approach to research. I will define what I mean by validity: Validity in this case means: Am I able to find out what i’m trying to learn from my design research? A more academic definition of validity and reliability can look here. Common pitfalls during research So what are common pitfalls unexperienced researchers make when out in the field? These mistakes happen quite fast, and don’t seem that impending at first sight. After all, you had your respondent answer your question right so what can be wrong? Well, what if the answer you got wasn’t a true representation of reality. As soon as you’d use those insights for your design you could be well on your way in building a unsuccessful product. Self-reporting error Self reporting error is a challenge in qualitative research. Kim Goodwin describes in Designing for the Digital Age (p. 55–56): It’s not out of dishonesty, but it is because our self-perception is not as complete and objective as we think. This leads to incomplete or wrong answers to your research. Priming & Anchoring Schacter (1999) described the seven sins of human memory. One that I find interesting is the principle of suggestibility. Suggestbility means that someone takes a recently offered bit of information and uses it to inform their memories and perception. Again, not out of dishonesty but because are brain likes to take fresh input and use it in given context. Two principles Kahneman described in his book: Thinking, Fast and Slow made me realise how we influence our research insights. By priming our respondents through priming and anchoring. For detailed explanations of what both are I suggest picking up Kahneman’s book. Priming and anchoring are quite alike, they have subtle differences but I can explain the concept as following. Priming is like planting a seed. As soon as a new seed enters our brain it starts connecting to earlier seeds, or bits of information they perceived sometime in their life. We as researchers might make the mistake of planting a new seed in our respondents brain. Our respondent’s brain starts taking care of this little piece of information. Relating this little bit of information to a distant memory with little significance, but because YOU just primed the respondent this is the first thing he or she can think of. Responding to your interview question with this information. But what if the true answer is quite different? What if the significance of the given answer is so little that it is worthless to put your energy and time in designing a solution for this specific insight? Lets say we are researching the world of a road racing cyclist. We are asking him about his frustrations: Researcher: “What are your biggest frustrations when going out cycling? maybe broken parts for example…” Cyclist: “Oh yeah, when my shoes won’t un-click from my pedals, that can sometimes be frustrating.” The researcher made a mistake here, he planted a seed. He suggested frustrations about malfunctioning bike parts. The cyclist is primed to think about incidents where some parts or gear on his bike were not working the way it supposed to be. In reality however, these incidents are quite rare. Seeing this cyclist buys high-end gear and rarely has experienced malfunction. His real frustration is other road users that disrespect his presence by blocking the road on purpose. But he didn’t think of this as his mind is primed by the researcher mentioning faulty bike parts. I understand that giving a single example to clarify what your asking will not compromise your entire research effort. Some people won’t be suggested that quick or primed by information but some might. Some solutions Are my questions open-ended, did I apply the 5W & 1H rule? The 5W & 1H rule means that any question should begin with one of the following words: What, who, where, why, when & how. It allows you to have a more objective approach and less inclined to push your respondents in certain corner. Do my questions or any other information im giving consist of triggers that might prime my respondent to think of any memories that are not important? And if so, am I aware of the purpose of this being here or is there any other way to present this? The most important solution is to triangulate your research. As we discussed with the self-reporting error, the human memory isn’t that objective. We can’t ensure a sealed & valid method for interviewing, as some priming may happen alongside the faulty memory of our respondents. That is why i’m asking you to combine interviews with other research methods. You could use your interview findings as a way to form a hypothesis, but none of this is valid unless you triangulate your research and collect data in some other way that confirms your hypothesis. This leads me to another common pitfall in analysing your data. The texas sharpshooter fallacy The texas sharpshooter fallacy is an informal term for the incorrect analysis of research data. The story of the Texas Sharpshooter is about a man who took his rifle and started emptying his gun aiming randomly at the side of his wooden barn. He painted red targets around clusters of bullet holes. The Texan showed the side of the barn to everyone in the region and they called him The Best Sharpshooter in Texas. The things here is that the targets, which is our hypothesis, got defined at a later point, after the gun was emptied randomly. Any spot where qualitative insights are close to each other, can be misinterpreted as a behavioural pattern for our target group. What if we were to paint the targets first, and fire again? Would we be able to get the shots still as close together as randomly emptying our gun? We simply don’t know! The solution is to triangulate. Use what we already know through rounds of research to form a hypothesis about our users. Use this hypothesis and try to confirm it through other research methods. Examples could be probes, shadowing, workshops, service safaris. The possibilities are endless. Prototyping your first ideas based on your earlier research and testing those is also testing your hypothesis. Be true to yourself and be ready to throw your prototype in the trashcan! For inspiration in research methods there are lots of resources: This Is Service Design Thinking IDEO design toolkit, IDEO Just Enough Research, Erika Hall. Great book for design researchers of all levels. Many others… Awareness What I am trying to make you realise that the first round of research is never enough for you to understand the humans you are designing for. they are influenced by daily triggers coming from all sorts of directions. Frequent research with different methods allow us to ensure that the insights we get are valid. Taking basic steps in the setup of your research allow you to reduce common made mistakes. On the other hand, do not try to build up a perfect academic approach to research. As designers we usually don’t have a lot of time and budget to be really invested in loads of research. This is why I find it extra important to be motivated in validating your research as much as we can. This way we make the best use of our limited time, allowing us to build better products. Thanks so much for reading my first article. If there is any feedback, ideas or rants you would like to share, please do! This way we can learn from each other. Have a nice day.