When I was at Zynga, we shipped a game that had marginal success. And in the deep dive the product management lunch lead said:
Engagement is really tough to move
At Zynga, Mark would demand game-changing features, demanding that we change the product in place and if we could pull that off the theory was we could change the retention curve.
And being an engineer, and surrounded by entrepreneurs, my assumption was that through the application big data and science we could change this number.
Then I had the opportunity to try and drive machine learning models into games to improve core metrics of the game.
The theory was that auto-tuning the game would improve engagement and retention.
And it worked, to a point.
And what I realized was that the hard problem is building a fabulous product. And a great product has high engagement. Everything else we do is about tuning or improving the excellent product at the fringes. And that changing engagement is equivalent to creating a new product.
And that got me thinking as to why that is impossible. And what I realized is that big data collects information about the product that is. And can only answer questions about what your product is doing.
To change engagement, you have to build a new product with new features and net new capabilities. And the data for that product doesn’t exist in any of your big data systems.
The short version of this story is the following, engagement is what it is, and if it isn’t what it needs to be, you need to scrap the damn thing and start all over.
I think that’s the wrong conclusion. Small scale changes tend to lead to small scale improvements. Auto-tuning numbers has it’s limitations in delivering improvements to user experience.
I would argue that a truly adaptive ‘machine-intelligent’ experience can be woven into a product at it’s core, and that’s where the next big step in interesting user experience will come from.
I would argue that weaving adaptive machine learning into a product that didn’t have it to begin with is changing the core product.
Consider twitter’s feed. Twitters feed never had a dynamic machine learning algorithm instead being a real time feed. Adding a dynamic feed would change the way you engage with the product. And I would almost claim it would change the fundamental nature of the product.
It’s about scope 🙂 Changing twitter’s algorithm is drastically impacting what users experience. Changing tuning numbers change the friction but doesn’t change the experience nearly as much. The analog to twitter’s change in say a match-3 game would be letting data decide not just how fast level progression is, but in what order the game should expose levels to users.