When you look at AI systems, they are really good at getting to an approximate answer. If you think about randomized algorithms, they don’t return optimal answers, but they do return good enough answers most of the time with a lot less effort. When a product manager wants to create a product using AI and isn’t an engineer, they can get a pretty good, approximately correct answer. But it’s approximately correct.
In fact, about 4 years ago, my buddy and I chatted about this. AI is clever, meaning it gets to an answer. AI is not correct, meaning it doesn’t produce the precise correct answer. Sometimes you need the precise correct answer, and that’s where the LLM fails.
The workflow between product management and engineering is going to look like this; PM provides an approximate answer that gives a good-enough view of what the feature is. The PM team uses the LLM to create a plan and to render the code. The goal is to gain line of sight into the feature and validate its value.
Engineering takes that feature, and then creates a better plan that is informed by engineering expertise, and attributes the PM team that is narrowly focused on the feature, doesn’t have to, and should not have to care about. The detailed plan is then used to render the code. The old world was PM produces a document, then engineering creates a plan, renders code, PM reacts to the code, PM changes design, and …
The new world is PM produces a prototype rendered to code automatically from a plan created by the AI tool. When PM is satisfied with the prototype, engineering reviews the tool and then creates a more detailed, precise plan to generate the code.
Engineering may use AI to create part of the plan, but the evaluation of the plan and its correctness will contain insight and details that PM doesn’t and shouldn’t care about. Is it possible for PM to go end-to-end? Yes, but I also believe that the less you rely on expertise, the more likely you are to experience poor outcomes over time.
Anyways, the way I look at it is this: Use AI coding assistants to get an approximate answer that meets your goal. Take the approximate answer and refine the plan until it is more correct. Use the corrected plan to render better code
Leave a Reply