My partner Albert has a great post up today on the limitations of data and A/B testing in managing a product. He strongly advises that product teams do two things that many don’t do
1) do in-person product testing sessions to see users interacting with the product and develop an understanding of why users struggle with aspects of the product
2) use the company’s product (really all employees should do this)
I want to echo Albert’s point and suggestions. I feel like the companies we meet with and work with generally do a good job of instrumenting their products and collecting on data on what is working and what is not working. But they often don’t have good answers for why the behavior they are seeing is happening. It’s hard to fix something you know is broken unless you understand why it is broken.
