Product lessons from Dan Robinson, former CTO of Heap

At Assembled, we’ve made it our mission to have a culture of continuous learning. We’ve been inviting the best minds from the startup space to share their wisdom on building successful products. A few weeks ago, we had the privilege of hosting Dan Robinson for a dinner discussion.

Dan was one of the first engineers at Heap and served as its CTO for almost 9 years. He helped Heap scale to more than 350 employees and a $110M Series D funding round, so he’s in the unique position to have seen a company scaling at many different points of growth.

Here were some of the top takeaways from the discussion:

• Execution, execution, execution: Most businesses come down to how well you can make a product that matches your user’s needs
• Talk to your users with intellectual honesty: Be careful with leading questions that can bias your users
• Taste the soup: Try out your own product as often as you can

The Burnt Pizza Problem

You need to determine whether the pizza is burnt (e.g. did you execute poorly) or if the pizza was a bad idea.

The default state for early stage startups is to have low, inconsistent usage. Thus, one of the most important jobs of anyone at a startup is to diagnose low usage and identify strategies for increasing it.

Dan uses the Burnt Pizza paradigm as one way to think through this. Let’s say you’re experimenting with a new food and you give your test users burnt pizza. They don’t like it. Was pizza a bad idea or was the problem that it was burnt? Whenever Heap shipped a feature that got weak usage, the right question was: “were we wrong about the user need or did we do a bad job solving it?”

What I found most interesting about this paradigm is that often many of the best companies have ideas that are very similar to a myriad of other competitors. Facebook, Myspace, and Friendster famously had very similar ideas for their offerings. However, Facebook grew into a generational company by focusing on execution of viral features and moving very quickly. They didn’t “burn the pizza” but instead focused on listening to their users.

Talk to users, but in a specific way

Which brings us to the next topic: how to listen to your users. YCombinator famously says you should only do two things at your startup: “write code [and] talk to users.” However, it’s really hard to talk to users in the right way and not bias them in the wrong ways.

Dan mentioned there was a product at Heap that didn’t work out. Heap had many internal meetings about how to build this product and how it would fit into Heap’s future strategy, and only once most of this had been figured out did they talk to customers.

This led to customer conversations like: “If you could have X, would you use it?” or “Do you think X is a good idea?” These questions are predisposed to cause your users to say nice things to you. A user can always say “yes, I’d use it” but they don’t have any skin in the game. You’re also not getting any useful information on how valuable your product is.

For Heap’s next product launch (which was much more successful), Dan ensured the team got very precise about how they would learn about what users wanted. He asked questions like “How would you articulate the value of this product to your teammate?” (helps you identify how the user thinks about the value of the product) and “what have you already done to solve your problem?” (helps determine how much of a problem it really is). He also asked for pilot users to pay, which is the true determiner of how valuable a product is.

Ultimately, Dan said this all came down to intellectual honesty. The best way to build a useless product for months is by letting your excitement influence the conversations you have with users. Instead, you should focus on trying to be as unbiased as possible on the most important problems that users are actually facing.

Create a culture of product learning and product focus

Dan recounted a story of Heap’s goal to have every single employee “taste the soup”. For the quarter, every single Heap employee (from engineers to salespeople to HR folks) was required to engage with the Heap product. Dan mentioned that the night before the deadline, he and the CEO were furiously direct messaging folks on Slack making sure they would finish this requirement.

I found that anecdote fascinating: here’s a CTO of a 200+ person company going out of his way to create a culture of product focus. When asked why, Dan pointed out that it’s relatively easy for the entire team to have a deep understanding of the product in the early days of a startup. However, as Heap expanded, maintaining this depth of knowledge and empathy with users became harder and harder. Therefore, a sustained focus on the product became even more crucial — it was the only way to get everyone aligned on how to create a great product.

Dan closed with this: startups ultimately have to build something that people find valuable enough to pay money for. It’s one of the most basic observations, but often the basic things are the most important.

If you’d like to join in discussions like these (or help build something that’s valuable enough to have companies like Stripe, Etsy, Zoom paying for it), feel free to email me — Assembled is hiring engineers and engineering managers.