The Product Quality Staircase

The (Software) Product Quality Staircase TLDR; Your Back-end can only be as good as your Front-end which can only be as good as your Design which can only be as good as your Information Architecture which can only be as good as your Prioritization of User Goals which can only be as good as your Understanding of your Users. Whenever you see a successful product, people rave about how well it’s designed, how fast the APIs are, how seamless the experience is, how much the customer never knew they needed it, and how easy this product is to market. But, when you are actually part of a team responsible for developing a product, you don’t know what success looks like. In a storybook, you’d hire or train the best Developers, Product, and Analysts in the market and magically the perfect product would emerge. In reality, you have a limited budget and you need to get something done in the next 3 months or your team and/or company are toast. You can’t do everything well. So where do you prioritize? How much effort do you put into each thing? Here’s the biggest secret in the tech world: the best developer in the world is useless if they are working for a bad product manager. Here’s a related non-secret: most developers are terrible product managers. So how is quality actually determined? Here’s a visual I call the Product Quality Staircase. A Product Quality Staircase (thanks draw.io!)Some takeaways: In each step, some quality is decreased from the potential quality it had. Every step may bring the quality down to zero: pure waste below it. Increasing the quality at any step requires the steps below it to be re-executed, and introduces new risk for quality decrease. The way to maximize quality is to minimize waste at each step. Optimizing a single step is not enough. So understanding users at the expense of quality somewhere else does NOT necessarily increase quality, but will increase potential quality. Almost every product is assumed to be of the highest quality. Revising a step (such as learning more about a user or re-prioritizing goals) means potentially needing to re-execute all steps below it. Also, there is an interesting management takeaway: Investing highly in the earliest steps is more efficient in the long term, but iterating quickly can reveal discrepancies in your assumed quality sooner and start to provide value to customers (and make money). Balancing iteration length and quality is probably most of management’s job. Iteration Sometimes, despite a quality bottleneck in a previous step, quality actually goes up. Why? Because the people in that step are doing the job of those on top (or by sheer accident). You can enable this in your organization by passing down the data collected a the top. Also, you should design feedback loops in your organizational structure so that when someone lower on the staircase sees something wrong, it informs the previous steps and flows down again in future iterations. This is the essence of iterative design and why iterative software development is so powerful — Each step is an opportunity to make the steps above it and below it higher quality. Steps In the Process In this section I go in depth to each step, who does it, and how they affect product quality. I also try to address common misconceptions about each step in a company’s asymptotic search for product quality (and thus, profit). There is no TLDR; on this. Each one is a gross over simplification anyway :) Understanding a User In this diagram above, the most fundamental cause of waste is misunderstanding the user. Misunderstandings have a few classic causes: Asking the user what they want. Asking a user what their goals and preferences are reveal more. Assuming you know what your user wants. You are NOT the User. Not validating assumptions. When we lack information, we make assumptions or rely on our biases. Most assumptions, such as old people can’t use computers well, are incorrect or may be incorrect for the user you have. Also, I’d put “validating” with non-users in this category, e.g. calling your family and friends. Any I Love Lucy episode episode illustrates one or more of the above perfectly. If you want a great book on better understanding your users, Try The User Is Always Right: A Practical Guide to Creating and Using Personas for the Web. Alan Cooper and friends’ About Face is also an amazing reference. The responsibility for understanding a user should be by everyone on the leadership, often executive, team. Often they hiring branding, market strategists, or UX professionals to help them ask the right questions, challenge their assumptions, and reveal insights they missed. But if the leadership team does not make it their #1 priority to understand their users, the company will do more poorly or outright fail. They are also responsible for making sure the rest of the company understands the users in the same way. Often over the course of a product, a new user will appear, perhaps because of an opportunity sales notices. When a new user appears that is not understood, the entire product staircase probably needs to be re-executed. This could result in a new product (if synergy is too low but profit potential high enough), modifications to an existing product, or deciding it’s not worth it to take the sales opportunity. Prioritization of User Goals All products help a user achieve one or more goals. Some of the early apple app design guidance stated this very clearly: do one thing and do it well. This is the easiest path to quality, because there is no waste. However, in real life, most people don’t want to install thousands of apps; hundreds are already enough. If we pick more than one goal, then there must be synergy between the goals. Sharing a photo at the same time as a message is synergy. Choosing goals is also difficult. You can make the best photo sharing app in the world, but if a user often wants to control who sees it, it doesn’t often make sense to open up a different application. However, every goal you add increases complexity and opens up the possibility for poorly prioritizing goals. For example, Facebook doesn’t want users to care about privacy because it makes the product more complicated and potentially lower quality. Every goal must have an explicit priority relative to another. Otherwise, the product will achieve each goal poorly. Even a Swiss army knife prioritizes certain tools over others. In the previous Facebook example, Privacy MUST take a backseat to sharing photos, messaging friends, and finding other people or their product will be lower quality. Sometimes goal priority can be dynamic; goals can have different priority for certain users or at certain times. The most common of these is adding a “power” user. Priority switches tend to be very expensive and hard to execute as they can greatly increase product complexity. Often, it’s best to design a separate product. For changing priorities at certain times, most software products just cheat by adding “alerts” or carousels, which usually diminishes product quality. The responsibility for prioritizing goals should also be at the leadership level, and is often informed by debate between departments. Product managers and business analysts are the professionals that know how to evaluate the pros and cons of different goal priorities. Often each department is defined by one high level goal. But beware, when building a multiple goal product, the prioritization of goals often leads to the prioritization of departments. This requires a lack of ego, and a shared trust that everyone is in this company together. I’d feel dishonest if I didn’t mention that there is an inverse relationship between potential product quality and market size. The larger and more diverse your users, the less shared qualities exist and the higher a potential for goal divergence. This adversely affects your ability to produce a *single* quality design for all users. Sometimes it’s best to split your market and develop two quality products. It can also save your company culture some beating when certain departments always take priority over another. Information Architecture Wireframes, the visual map to an information architecture, may be the most underrated tool in software product development. People often think of them as the first step in the design process. On the contrary, they are the last step in strategy before implementation. After this point in the funnel, changes become very expensive and everything you produce becomes a liability. If you get it wrong, which means the user is unable to most easily achieve the highest priority goal and execute the other goals in priority order, then anything you do after this is wasted effort. Additionally this is where organizations often have turf wars. Instead of arguing at goal prioritization time, departments will argue at IA time that they need higher prioritization. This is good if and only if it changes the goal prioritization and does not bloat the wireframes. One of the biggest mistakes made when developing wireframes is requiring approval of multiple departments for them. This is the surest way to bloat and destroy a product. At this stage, the question should be “does the wireframe allow the user to achieve all goals and makes the most important goals proportionally easier”. You’ll also save a TON of time if you can get different departments to argue at goal prioritization time instead of IA time, though in reality most organizations wait until this stage to debate. The most common victims of bloat are large multifaceted organizations such as banks, universities, and hospitals. This is because they have so many different users with often conflicting goals. The challenge in doing IA for these organizations is balancing the importance and number of each type of user (personas if you got em) and the importance and urgency of each goal for each one. I’d recommend asking them what their phone tree looks like or training manuals for their front desk to do this effectively. If you’re doing this, good luck! There’s a reason Simple Bank is so effective. The responsibility for information architecture usually belongs with Product managers and UX professionals. Product managers are responsible for ensuring the previous goal priorities are justified and verifying the wireframes designed by UX professionals. Often, I’d say that Product Managers handle the churn of talking to everyone and collecting information while UX professionals are focusing on how best to structure the data and competing goals. This is why UX professionals working closely with Product managers often leads to high quality products. The most common problem with wireframes is not considering the mood of the user at each step. Wireframes should be careful to understand the mindset of a user when they start and finish a goal when placing two goals in the same frame. Technical Products When people think of technical products, they usually mean products that have a high dependence on speed, scale, and reliability of data. They may have a thin veneer of front end (such as google), but a majority of the work is in the back-end. Some products don’t have design or front-end requirements, as the user is another back-end or front-end. In this case, wireframes are usually designed by System Architects and the differ a little bit. System architects specialize in understanding the needs of those that consume the back-end services. Instead of considering the mood, they consider the intent and the relative proportion of goals. Determining how the services will be used is a major factor in how much effort the back-end will need, and usually involves a similar goal breakdown. Design Design is strange because it is both overrated and underrated. It’s overrated because most people associate the previous steps of Information Architecture, Goal Prioritization, and Understanding the User into this step. It’s underrated because most people don’t understand that this is where the feel of the application is defined through visual design, microinteractions, and implementing the spirit of the wireframes (map of goals, priority, and information organization). Design is inherently emotional and this emotion shouldn’t be underrated. The responsibility for information architecture usually belongs with a Visual Designer (which may be called a UX Engineer or something technical). They are creative professionals and often the quality they produce and their productivity are greatly affected by churn (meetings and interruptions). They are experts at finding constraints and empathizing with the user. Front-end Front-end is the technical part of design where it becomes interactive. If Front-end didn’t require so much expertise and wasn’t changing so quickly, it wouldn’t exist and would be part of design. When it is accomplished by the same person as the design, they’re called unicorns and usually cost a boatload more than a designer and front-end specialist separately. Their sole responsibility is to execute the design as closely as possible while balancing quality and speed. Often, the speed or limits of technology will limit some aspects of a design; Quality is purposely reduced to lower cost. If a programmer decides to change a design without knowledge of the goal priority, reasons for information organization, or a user’s goals, mood, and feel, then quality “waste” occurs. The responsibility for the front end usually belongs with a Front end Engineer or App Developers. In the process of making a design interactive, they are responsible for consuming the data they need. They are also creative professionals and similarly affected by churn. They are experts at balancing technical constraints with design goals. Front-ends come in 2 primary flavors: browsers and devices. Most front-ends don’t have much synergy; making a front-end for a different browser and device costs the same as if the other front-end didn’t exist. There is a huge push towards technologies that increase this synergy, such as React Native and Electron, but they are very very new. Back-end Finally, most software needs to persist data or provide some kind of central coordination. The back-end is responsible for defining and controlling access to data as well as determining what and when things happen in the background. In terms of product quality, back-end really only determines one thing: how fast and reliably they can provide and mutate data. Often, services become more complicated when the number of users that interact grow. The back-end is often created by a Software Engineer and teams may have more specialized roles such as Data, Operations, or QA Engineers. They take data needs from the front-end and partners, make services available, and maintain those services. In the interest of speeding time to market, organizations often define the data they think they need before or during implementing the front end, which can create wasted effort as well as buggier/bloated code. There is currently development occurring on making this parallel development more efficient, mostly being done by Facebook’s GraphQL product. Disclaimers and Call for Feedback I don’t have an MBA. I am not a business expert. I don’t actually know if this idea is common knowledge, revolutionary, or just plain wrong. This is only based on my experience and what I’ve seen work well, or more often than not, where I’ve seen things fail. So, if there’s a better idea out there or I am clearly misinformed, please leave a comment! I mean, really, there’s not much better than learning in public.