When I was at Zynga, we shipped a game that had marginal success. And in the deep dive the product management lunch lead said:
Engagement is really tough to move
At Zynga, Mark would demand game-changing features, demanding that we change the product in place and if we could pull that off the theory was we could change the retention curve.
And being an engineer, and surrounded by entrepreneurs, my assumption was that through the application big data and science we could change this number.
Then I had the opportunity to try and drive machine learning models into games to improve core metrics of the game.
The theory was that auto-tuning the game would improve engagement and retention.
And it worked, to a point.
And what I realized was that the hard problem is building a fabulous product. And a great product has high engagement. Everything else we do is about tuning or improving the excellent product at the fringes. And that changing engagement is equivalent to creating a new product.
And that got me thinking as to why that is impossible. And what I realized is that big data collects information about the product that is. And can only answer questions about what your product is doing.
To change engagement, you have to build a new product with new features and net new capabilities. And the data for that product doesn’t exist in any of your big data systems.
The short version of this story is the following, engagement is what it is, and if it isn’t what it needs to be, you need to scrap the damn thing and start all over.

