How do you handle conflicting or contradictory user feedback?

At Amazon, customer obsession is a key leadership principle that helps us understand our users. We start by listening to their needs and pain points through extensive research, including interviews, surveys, and user testing. We then develop products that meet their needs and exceed expectations, trying to solve their specific jobs-to-be-done. By prioritizing customer obsession, we ensure that we deliver valuable products and exceptional experiences.

When getting contradicting feedback, it's important to understand the context first! Try to understand the context in which the feedback was given. Is it a one-off comment or a recurring theme? Was the feedback given by a representative sample of users or just a few vocal users? Understanding the context can help you assess the credibility and validity of the feedback.

The best "method" is to ask questions and listen. When the customer is giving me conflicting reports usually it's because they either don't want to tell me what's wrong or don't know what's wrong. Their conflict is emotional. Sometimes they are just frustrated, and sometimes... they are looking at the competition. Yes, validate but reinforce the partnership. Ask "What is happening with you/your team/your company because of this?" then listen. "If we could solve <this> are there any other barriers to this being a success?", then listen. If this is the only issue they state, then you probably have the right problem. If they add "Well..." now you are probably getting something closer to the real problem... listen.

When confronted with conflicting user feedback, approach it with open-mind and appreciation. Avoid forming early biases, even if you disagree with the feedback. Value the user's input and be receptive to their perspective. Next, Segment the feedback based on demographics, usage patterns, or sentiment to identify common patterns and trends. Then prioritize the feedback according to its reach, urgency, and impact. Analyze the feedback for patterns, trends, and outliers, and take appropriate action to enhance your product for all users. By following these steps, product teams can effectively manage conflicting feedback and continuously improve the product.

The real question to be answered here is - As the Product Manager, do you really know who your users are?.... thier characteristics, income, finances, market, demographics etc. Once you do, you would be able to get the right and non conflicting feedback. In the case there exist, it simply means the users are not really your core users, they don't understand or know what the problem is, they are too emotional to state the problem or you are not asking the right questions. Once users are confused and giving conflicting answers, guide, redirect and ask the right questions.

In my experience OKRs encompass KPIs and it’s best practice to use SMART goal principles for KRs. Click the💡 below to agree.

Evaluate the feedback based on its relevance, feasibility, and impact. Relevance refers to whether the feedback is relevant to your product goals and objectives. Feasibility refers to whether the feedback can be implemented given your resources and technical constraints. Impact refers to the potential impact the feedback could have on your users and business goals.

Multiple sources is critical to ensure that you can get a broad view of user feedback, and it also reduces bias. When analyzing user feedback across multiple sources, consider using GenerativeAI (i.e. ChatGPT, etc) to help summarize, organize into topics, and highlight outputs for a large data set (like App store reviews). This can help fast track analysis that you'd need to do manually otherwise, and can help with presenting the information to the broader team.

Have you ever been in an argument or heated discussion and later thought about all the things you would have said? Well, users are no different. When receiving what may seem like contradictory or conflicting information, I try to find patterns across the various ways we collect feedback. Sometimes that means comparing interview insights with analytics to look at how users think versus how they behave. Also, different feedback collection method can illicit different responses. Imagine getting an NPS survey while frustrated on a site versus 2 months later during an interview after accomplishing the task — you’re in a completely different headspace. So, relying on one single method can never show the whole picture — humans are complicated.

Validate the feedback by testing it with a representative sample of users. Conduct user research, A/B testing, or usability testing to gather data on the effectiveness of the feedback.

In my experience, it's always better to look at classifying and the distribution of the feedback rather than treating all feedback the same. Decisions should be made considering the distribution of the feedback (ie, what is 80% of users saying). This helps to filter out outliers and noise. Occasionally you will have a single data point but high impact - think about your highest paying customer the only one screaming about a capability gap in the product. This is then a conscious trade-off considering the churn risk and its business impact.

As much as practically possible, utilize spontaneous, qualitative and continuous feedback 💡 Overall, from my experience, it is important to have continuous yet spontaneous feedback discourse with Stakeholders. The problem with planned or structured feedback sessions is that your stakeholder would be inclined towards giving an "ideal case" feedback rather than feedback that stems from their "reality". Qualitative feedback goes a long way in gathering exactly how the stakeholders feel about your product/solution, you get this from listening to what they are saying and are not saying. A lot could be hidden in the quantitative feedback, mostly from surveys. I find that stakeholders love to express themselves, so why not leverage that?💡

Now two user personas are exactly the same, and not everyone will have a positive perspective. Therefore it is important to get an accurate sampling of categories of personas and measure percentages of responses to be able to develop a marketing strategy.