I have been building with Claude Code for nine months now, and I thought I had understood how memory and context worked. I do understand the concepts, don’t get me wrong.

A while back, I was noticing that Claude wasn’t saving the notable parts of the conversation into memory and seemed like it had amnesia. So, I built a skill ‘/save’ which basically asks it to identify key decisions from a conversation and save them to memory and create any relevant context files. Over time, as I learned how it was doing, I got into the habit of asking it to ‘validate’ its work before I exited the conversation.

Memory.md has a 200-line, 25KB file limit. Since Claude Dreaming came out in May, I created a rule that I would do memory cleanups in one go when no other conversations are ongoing. Recently, Claude had been flagging to me that I was reaching 175 lines and that I should clean up the memory. So a week ago I spent a chunk of Saturday afternoon cleaning up outdated parts of the memory (completed projects, irrelevant references, feedback consolidations, etc.). After last week’s cleanup, Claude kept reminding me that memory at 181 lines needed to be cleaned up (again!).

In addition, I was starting to feel brain-fry reading AI-written content for hours, and my writing-style rules (for Claude to edit my writing) had drifted. That rules file was also 35KB, grown over time.

I wondered whether I had misunderstood how memory works or if my setup was right. So this past weekend, I sat down to understand how to optimize memory.md. I started with the basics. Refreshing my own memory (pun intended) on how context and memory work and best practices to engineer them. The guidance on how to use claude.md and memory.md has been updated as the technology has evolved. Mine was operating from the initial setup. I didn’t just want to clean up or change the harness. I wanted to make it better. So, I looked at Anthropic’s own guidance on best practices.

Here are my lessons:

1. Audit for conflicts: Over time, Claude had written rules (sometimes conflicting) in multiple places, so it was referencing stale information where it wasn’t updated. My writing-style, LinkedIn-rules, blog-writer, and humanize skill all had a different levels of detail in my writing voice, operational rules like publishing cadence, and AI slop to avoid. So depending on how I phrased my ask, it would invoke a different rule. Fix: If possible, write once and point to it from other places. One writing-style, cleaned up with my original writing as examples, one humanize skill that points to the writing-style, and one operational file with platform and publishing mechanics - kept separate with purpose.

2. Audit what doesn’t get flagged: The system reliably flagged that memory was reaching its limit. But instead of flagging that the writing had drifted, it updated the writing-style with feedback, and that drift increased the file size with conflicting information in the same file. I found out through brain-fry after hours of trying to comprehend the AI-written content. Fix: I added a ceiling to the writing-style to not go over 220 lines, which will force a cleanup next time it starts to drift.

3. ‘/Save’ identified key decisions and wrote them accurately. It also had a step to “update a file,” but not all files with relevant references. My validation habit checked that things got written, not whether the old thing stopped being true. So I updated the step to look for all references.

4. When I ran the new cleanup again, the gate accurately loaded my writing-style for drafting but not reviewing, and Claude reviewed my voice without actually reading my voice file. So the narrow trigger had to be updated to cover both reviewing and judging.

The verdict on whether this worked is still pending. At least Claude isn’t reminding me that the memory is reaching a limit. My memory.md file now reads at 58 lines.

Stale memory is worse than no memory. Append-only memory scored 0.210 against a no-memory baseline of 0.309 under drift, and “falls below no memory because the stale precedent remains active.” As I discovered in my own setup, my /save was appending selectively, and it cost me two Saturday afternoons.

Someone asked me that last week, and it isn’t the first time I’ve been asked it.

It’s a fair question. It’s also a logistics question.

Then I watched someone give a demo who had been using the same conversation thread for weeks to produce a report, loading the context of previous turns before resuming, and burning tokens the whole way. The context window was full. The evals were far from reality.

We are asking people to use AI to do the work and produce results. Increase efficiency. Reduce cycle time. Generate documents. We have selectively given tools to people to use. We have not given them any of the necessary training on how to understand and manage a context window, guidance on when to start a new conversation, or how memory persists between them.

That costs us real dollars in inefficient token usage. It also degrades the output, and people hit limits mid-task without understanding why. This tells me that they have not been taught how to think with the tool.

Then there are the people who are paralyzed. They have a license to a popular coding tool and have never opened it.

This happened to me too. I was brazenly using Lovable on my personal account to prototype ideas, and then I got a corporate license and froze. On the work account, I was waiting for inspiration, for the perfect use case that would show my best work. I was performing excellence. On my personal account, I could just build random things for a user of one.

So I wonder how much of this is really a training problem. We are measuring who is using it and who is not, and usage is the one number we can track without changing much else. We are not looking as hard at our org structures, processes, incentives, or at defining what good looks like with these tools. We have not permitted our teams to slow down on the judgment parts so they can go fast on the execution parts.

So we ask the logistics questions: which MacBook, which model, which coding tool is most efficient. We skip the judgment questions: When should a model handle a task? How would I know if the output is any good? What context is passed between one conversation and the next? And, when do I edit the markdown file myself vs. asking a model to do it?

Those are the questions I’d want us training people on. What gaps are you seeing: training or judgment?

I want to be upfront. I am not a lawyer. I have worked in a call center and have designed multiple applications that support the ecosystem. So I am looking at this from the human and design side, and leaving the doctrine to people who know it. This is purely a thought piece.

Four parties and no obvious defendant

Let’s start with who is involved when a customer’s AI agent interacts with a human agent.
There is the customer, who instructed their AI to achieve an outcome. There is the AI agent, built on a model by a provider to follow instructions. There is the human agent who is handling the escalation path. And then, there is the employer of the human agent. Sometimes the employer is a vendor to a company whose customers the human agent is handling.

The human agent has no binding relationship with anyone else except their employer. The agreements that allocate risk in agentic interactions are between model providers and their users, and the recurring point in all of that writing is that only parties to an agreement can invoke it. Someone who is harmed but is not party to those agreements generally cannot reach the protections inside them.

Protecting Employees

Employers already have a duty to protect employees from harassment by anyone who is not employed there, where it is based on a protected characteristic. Under the EEOC standard, an employer can be held liable for how a customer treats an employee. Two things have to be true: the employer knew about it or should have known, and did not act to correct the situation.
The law is not written with AI agents in mind. It was written for the customer who screams at someone for twenty minutes, and it exists because we decided that a person should not have to absorb that as a condition of employment.

I want to be careful about how far this stretches, because the version of this argument that overreaches is easy to dismiss. As I understand it, harassment in the legal sense generally requires a protected-class basis. An AI agent that is simply relentless is not harassing anyone in that sense. It may even be programmed to be polite while it does it. None of that makes the call any easier for the person taking it.

An employer sets the conditions of the job, which means an employer owns them.

Methods and Procedures

This is a design problem.

The human agents are measured on handle time, resolution rate, customer satisfaction, and adherence to policies. Every one of those assumes the person on the other end eventually gets tired, or satisfied, or gives up. That assumption has held for as long as contact centers have existed. An AI agent does none of it. A human agent who draws a queue full of these conversations will watch their numbers drop, and nothing in the measurement will show why.

I was once having a rough day on the floor. Almost every call that day was harsh, and then one got disconnected while the customer was mid-rage. My supervisor pulled me aside to ask if I had ‘hung up’ on him. I had not. I got dinged for ‘potential disconnect’ anyway.

The policies dictate whether a human agent is allowed to end a conversation. Most contact centers have a disconnect policy for abusive callers, and without an explicit authority, people stay on the line for fear of it being scored against them. Those policies were written for humans. A human disconnects eventually. An AI agent does not.

So what should employers do?

There are a few options:

1. To start, have training and policies around identifying agentic interactions and what a human agent can do.

2. Give explicit authority to end the conversation when the AI agent is borderline abusive, and don’t score the human agent poorly for that interaction.

3. Determine how agentic conversations will be scored against a human to avoid penalizing people for conditions they cannot control.

4. Continue conversations with human agents as the tech landscape evolves.

And, I can’t emphasize the last point enough. Asking the people doing the work what they are seeing is the single most important way to understand the agentic behavior against human agents.

The uncomfortable version of this is that the employer has the duty and also owns the metrics creating the pressure. The duty and the scorecard were both built for humans, and need to be revisited for AI agents.

If you run one of these organizations: do your human agents know what they are allowed to do when it isn’t a person on the other end?

If you have shipped a customer-facing bot in the last few years, you probably had a disclosure conversation with legal. The disclosure that a user is talking to AI (or AI may be incorrect) is a common placement in the user experience. It became standard well before it was strictly required, because the alternative looked bad and nobody wanted to defend it. That instinct was right.

What I can’t find is anyone applying the same instinct in the other direction.

The rule was written with one person in mind

The EU AI Act’s transparency rules require that systems interacting directly with people be built so those people know they are dealing with an AI, and those obligations start applying in August. California has had a bot disclosure law since 2019, which covers use of a bot to mislead someone about its artificial identity in order to incentivize a purchase or sale, or to influence a vote.

I want to be upfront: I’m not a lawyer.

These rules paint the same picture. There is a consumer, and there is a company with a machine, and the worry is that the company’s machine will fool the consumer into buying something or believing something.

That’s a real worry. But are the people drafting these rules picturing a person on a customer service floor, on the receiving end of a customer’s AI agent that was told to get a credit and will keep working until it does?

Are the human agents already covered?

The EU provision is written as an obligation to inform natural persons who interact with the system. It doesn’t specifically say consumers. A customer care agent is a natural person. On a plain reading, the provider of that customer’s AI may already owe the human agent the same disclosure the law demands in the other direction.

I haven’t seen anyone build for that, either as a norm or a product requirement. The obligation may exist on paper, but do the systems people actually use reflect it?

I wonder if the disclosure to human agents is a design problem. Agent-facing tools have fields for account information and sentiment, and they presume an authorized customer is the one contacting them. None of them have a field for who or what is typing.

More importantly, if a human agent learns that they are interacting with a customer’s AI agent, what are they allowed to do differently? If they don’t have autonomy or at least methods and procedures telling them what to do, then the disclosure is just theater.

• Can they ‘disconnect’ from the AI agent to avoid being harassed by it?

• Can they escalate/transfer to a supervisor?

• Can they sue? If so, who? On what grounds?

The visible asymmetry

We built the bot disclosure quickly, ahead of the law in some cases, because the reputational risk was obvious and the person to protect was a customer.

The logic applies to human agents, but the asymmetry is on the pressure, because the person on the receiving end is an employee. And employees don’t generate the kind of risk that changes roadmaps.

I’d rather we notice that now, while the volume is still low enough to design for, than after it becomes the normal condition of the job.

What would it take for the person taking the conversation to be told what they are talking to?

Someone had posted a video of their own AI agent negotiating a bill with a company’s customer service agent while also filing a complaint with a government regulatory body. The poster gave instructions to the agent about the desired outcomes, and the AI agent simultaneously and autonomously interacted with a human agent while filing the complaint.

I am not surprised that the agent existed; I build AI agents myself. We have all been watching these get better for two years. What got me was that the AI agent picked a strategy. It made a call about how to achieve the defined goals, and then went and did just that.

Which means that at the receiving end of that conversation was a human being, talking to a customer’s patient, well-briefed AI agent that was running a plan and representing someone else’s interest, like an attorney. We have spent years designing for a human on the other side of that conversation, the one who is frustrated, or in a hurry, or genuinely wants to resolve a billing issue. That person is still there. But our experiences have not evolved to accommodate their (AI) representative.

It also doesn’t get tired or embarrassed, and it doesn’t take no for an answer. The training, the scripts, and the scorecards at the human end of that conversation were built around a counterpart who does all three.

Then there is a question about the disclosures that’s still unanswered. We’ve mostly settled that people should be told when they’re talking to an AI, or that it can make mistakes. The reverse isn’t settled at all. If a human agent is now talking to someone’s AI, is that human owed the same disclosure? I think so. But who would enforce it? You also can’t disclose what you can’t detect, which is where this stops being philosophical.

How would we even know? None of the usual channels would catch it. This won’t show up on a dashboard, in a test case, or a flag anyone thought to build. It surfaced because a person was pleased enough with what they’d built to show it off in public.

The human agents already know

In the early 2000s, I worked at a Hewlett Packard call center. Back then, if you called because you had an issue with your HP personal desktop computer or a printer, I may have taken that call. During one particular Thanksgiving, HP had launched a new line of PCs that turned out to be faulty. That weekend we were flooded with calls about new computer issues. The guidance we were given was to ask the customer to return it for a full refund. But we, the agents sitting on the floor taking calls, noticed the pattern early and knew about the issue with the computers before the company did.

The people taking the calls notice pattern shifts early, long before it’s a number on a dashboard. They notice the conversations that start going strange. We have all gotten good at spotting the AI slop tells by now. Someone handling chat all day would pick up on the tidier words faster than most of us.

So the knowledge tends to exist, in some form, on the floor. What’s usually missing is a feedback loop to carry it to the people who could act on it.

The gap in the feedback loop

The gap is that often talking to the frontline is treated as research. Something you do once, at the start, to inform a design. A study, a set of interviews, a ride-along that gets written up and then stops. Observation is a one-off by construction, and the advice to go “do more of” it is aimed at a person who may or may not be there next quarter, which, if we’re now designing for a class of user that arrives with a plan and doesn’t announce itself, feels like the thing to get right as part of product discovery.

Evals have become their own discipline in the last two years, with teams and even companies dedicated to watching what these systems produce. That discipline is getting better quickly. What it watches is the output. It doesn’t hear from the person who has been on the receiving end of forty strange conversations this week.

What’s the solution?

The shape of it seems to be:

• A standing channel from the frontline. A scheduled conversation where the human agents share with the product team what’s gotten weird lately.

• A named owner for the signals nobody is assigned to watch. Public posts, sub-reddits, community forums, the places customers brag about what they built.

• Detection treated as a design input. Whether we can tell a bot conversation from a human one changes how we design the evaluations, and right now most of us are finding out by accident.

None of this is exotic. It’s mostly a decision to treat the loop as part of the product, with the same ownership and maintenance as anything else we ship.

The customer’s AI agent in that video was better prepared than most humans who chat in. It knew what it wanted, and it had a plan for getting there. Plenty of people are working on how to design for that now. I am curious to see how the design practice changes as a result.

If you work on these systems, what would it take for the frontline to be where a change like this gets noticed first?

I have seen teams where several roles split the coordination work between them. One person translated what the business wanted. Another turned that into features and user stories. A third kept the engineers unblocked. You can’t collapse that into a tidier ratio just because it looks top-heavy, because each role was doing a genuinely different job. But none of them owned the one thing that mattered most: the judgment of whether we should build the thing at all. The coordination was well staffed. The decision wasn’t staffed by anyone.

The problem is not that these roles exist. Coordination is real work, and the connective tissue that translates between business, build, and holding context across hand-offs is often the most undervalued judgment on the team. That is not the part AI touches. What AI absorbs is the mechanical output some of these roles have been reduced to producing: the decomposition, the user-story drafting, the status reformatting. If a product manager’s job had been narrowed to turning requirements into tickets, without the judgment of why we’re building, whether we should, or how we’ll validate it, then that narrowed job is exactly what AI now does in seconds.

Across enterprises, product and engineering leaders are resizing their teams for AI. Companies everywhere are laying people off in AI’s name, hoping to gain efficiencies and do more with less. And so the same question comes back, now with real stakes attached: what is the right ratio when AI writes the code?

The implicit assumption underneath that question is that team design in the AI era is a sizing problem. But the way I see it, we gravitate towards what we can easily count. Headcount is easy to count. Capability is not.

The PM-to-engineer ratio is a vanity metric. It counts what’s easy and hides what matters.

Two teams with an identical one-PM-to-five-engineers ratio can ship two completely different things. The variable is judgment density, the capability the ratio can’t see. A team can have the perfect ratio and still be the wrong team. An organization can be rich in headcount and rich in judgment, framing the problem well and deciding what not to build. Or it can be rich in headcount and stuck in feature-factory mode, waiting for requirements and shipping to a predetermined date. The ratio tells us nothing about which one we have.

AI is very good at producing that output faster than ever before. A right-sized feature factory using AI tools just becomes a faster feature factory, and it may ship the wrong thing sooner.

So the move for leaders is not to right-size the team to some tech creator’s generalized ratio, but to raise the judgment bar and build the right capabilities. Only then should we talk about reducing headcount. Density doesn’t show up on a spreadsheet, but it is readable. Here are the questions I’d ask to tell whether you have a headcount problem or a density problem:

• Is each role on the team an order-taker or judgment-rich?

• Where are the real gaps and bottlenecks?

• How many hand-offs does it take for a requirement to travel from business to build?

• When did the PM last do discovery with real users?

So we, as leaders, need to stop asking what the ideal ratio is and start asking how much judgment sits behind each unit of build. A number on a spreadsheet is not the same as a team that can make sound product decisions. And judgement is much harder to see. Its density is what decides whether AI makes you faster or just faster at being wrong.

We all have the same models. The 5% have the clarity and specificity about the problem they are solving before spending a single token. A mandate to use AI is a direction. The clarity comes from explicitly defining the use case, the problem it solves, and how to measure success. A vague direction, without clarity of the problem and what success looks like, cannot produce a measurable ROI.

Here’s how we have been able to show return on investment:

1. Being clear about the use case and going deep on a problem: This pre-dates Gen AI. The use cases my team and I selected were specific problems we needed to solve. We got there by going deep on the problem before selecting the tool. That clarity applied to problems like accuracy of responses, the ability to search, translation of information, or intent classification. We worked to gain clarity from the ground up.

2. Making Build vs. Buy decisions: Early on, almost all vendors were solving the problem of summarizing a chat transcript. When the vendor market is flooded with solutions, it is essential for us to make build vs. buy decisions based on our unique context, compliance, and regulatory understanding. We could solve for what’s core to us, but didn’t make sense to rebuild something that we could buy off the shelf.

3. Aiming for boring use cases: AI output can be demo-friendly, pretty, and polished. The real returns are less glamorous in back-office automations, cutting the cost of manual efforts. The ROI comes from understanding where actual value is, without the flashy scenarios.

4. Simplifying the workflows: We chose to strip out the bells and whistles from the solutions, simplify the workflows, and go to the first principles of the problem we were solving. With AI, the solutions can seem limitless, and an appropriate tool choice for simple workflows can help unblock the teams.

Notice the patterns

Every one of these is really the same move. Know the problem before selecting the tool or solution. Be clear about the pain, the user, and what a good outcome looks like. Then AI can help get us there faster. If you think about it, none of this is new. It is the foundation of good product work, the part we have always known and sometimes skip when the pressure is on to just move at speed or scale.

Which brings me back to where I started. The AI mandates hand us the tool and tell us to go fast. It never says what for. The clarity is the piece we bring that separates the returns from the noise.

So the next time a mandate tells you to use AI, a more useful question might be ‘use it for what?’

And here is what the mandate has us all doing. We are burning tokens to see what sticks. We throw AI at the first pass of the PRD, the research synthesis, the email drafts and summaries, the presentation decks, and everything comes back looking good. The activity and the output are real and make us feel like we are making progress. Is any of it ‘good enough’ to endure the test of whether we are solving a real user problem? I am not sure we are focused on measuring that.

The number nobody wants to sit with

A few months ago, MIT’s Project NANDA 2025 report, The GenAI Divide: State of AI in Business, looked at enterprise generative AI and found that 95% of organizations are seeing no measurable return on what they have spent, and that is after billions of dollars poured into these initiatives, with nothing measurable to show for it.

The lead author, Aditya Challapally, stated that the failures had very little to do with model quality and almost everything to do with how organizations were trying to use them. He called it a ‘learning gap’, the gap between what the tool can do and the workflow it lives in. The pilots don’t scale well because they have to integrate with existing enterprise systems.

The mandate measures the easy things

The way I see it, the mandate measures whatever is easy to count. Are people using AI? Are we delivering more user story points? Has our velocity increased? All of that is activity. None of it tells us whether we delivered value with all this activity. “It is faster and cheaper” is a different sentence from “it is good,” and a mandate built on speed and cost will happily reward the first while ignoring the second.

That gap is the whole story behind the 95%. We measure the burning of tokens that shows activity, but never set the bar on whether the output was equal to the outcome (hint: It isn’t).

The misalignment was always there

There is a second cost that’s not on the velocity dashboard. When every team burns tokens against the same vague instruction, we do not get more aligned. We get more output, pointed in different directions. Each team builds its own artifacts with its own copilot and never reconciles them with the other teams. So silos deepen. Everyone is optimizing for what is getting rewarded, and right now what’s getting rewarded is more AI use.

Ovetta Sampson said, “Generative AI reconfigures the past and calls it the future.” That is exactly what an AI mandate without clarity does. We take the dysfunctional way we already work, automate it, and call it transformation. The misalignment was always there. AI just lets us produce it at a scale and at a speed we could not reach before.

What worries me most is the slide from outcomes back to outputs. We spent years in product trying to measure whether the work mattered, not just whether the work shipped. A mandate to use AI, measured by usage and speed, pulls us straight back to counting outputs.

So what are the 5% doing that the rest of us are not? That’s next.

So when the head of the Catholic Church wrote a long letter about artificial intelligence, I was curious, mostly about what he’d say about the technology, less about the doctrine.

Not about the technology

The pope isn’t a technologist and isn’t talking about the models and their benchmarks. He isn’t writing about product teams either. The further I read, the more I thought about the argument I have been making about product organizations.

The encyclical covers far more than this: AI and warfare, the dignity of work, truth, and the worth of a person. I am pulling one thread, the one about how responsibility gets distributed. The question under it is older and simpler: do the systems we build around a powerful new tool serve people, and who holds the power? To answer it, he uses two scenes from the Bible.

The first is the Tower of Babel. People decide to build a tower tall enough to make a name for themselves. One language, one direction, everyone aligned. It looks impressive, but it rests on self-sufficiency and sameness, and it ends in confusion and dispersion. The second is the rebuilding of the walls of Jerusalem. A city sits in ruins, and the walls get rebuilt. Nehemiah, leading the rebuilding, gives each family a section of the wall, listens to their concerns, and coordinates the work, instead of handing out solutions from the top. The city comes back through shared responsibility instead of one person’s heroics.

Same failure in product organizations

I have spent the last year writing about that exact failure, just inside companies instead of cities and countries. We wait for one person to be brave enough to say the requirement doesn’t make sense, to stop the experiment that is really just a launch, to admit they don’t have the answer. We build organizations that quietly run on individual heroics, then act surprised when the heroics don’t scale and don’t survive the person leaving. I have been calling the alternative structural courage: build the system so the right thing is the normal thing, and nobody has to be a hero to do it.

The pope refers to the principle as subsidiarity. In plain terms: responsibility should sit as close as possible to the people doing the work, and the levels above them exist to support that work, not to swallow it. When a higher authority pulls every real decision upward, it doesn’t just slow things down. It tells the people closest to the work that their judgment doesn’t count, so they stop offering it. That is almost exactly what I watch happen on teams. People’s judgment gets overridden, and what looks like a lack of courage is really a lack of empowerment.

He pairs that with a second idea, solidarity. He says, “The future of each individual is connected to the future of all.” For organizations, distributing responsibility downwards isn’t enough on its own. When people operate in isolation, everyone turns to protecting their own corner. In my own writing, I have called that density: one brave team is a rounding error, and a behavior is only sustainable when enough teams and leaders behave that way. Distribution falls apart without connection.

The pope also talks about the common good as a set of conditions that lets people do well. I understood this as a version of what I called ‘enabling constraints’. A good ‘enabling’ constraint doesn’t tell people what to do. It shapes the conditions so the right move becomes the obvious one.

Where AI makes it worse

A handful of large AI players have changed the conditions the rest of us work in. They reward what gets produced and how fast, not the judgment behind it. I keep noticing that most organizations adopt AI the way they adopt everything else: they measure speed. Organizations are spending hours discussing how to adopt AI faster, who gets to build with it, and who gets to decide what to build. We are shipping faster, and a lot of the time, we are shipping the wrong things faster.

The encyclical names the same worry. It warns against treating people as more valuable when they are efficient, until a person becomes a means of getting results. That is what happens to a team measured only on speed.

I have spent years building with AI, so the tools don’t make me nervous. You can hand a team the best AI available, but if the system doesn’t change, people optimize for what is being rewarded. And right now, what is being rewarded is speed: speed of delivery, speed of showing that we are using AI, speed of showing leaders we are thinking about it.

So structural courage matters more right now. If the questions your system asks stay the same, AI just helps you answer the wrong ones faster. The tool was never going to supply the courage to ask a better question. That has to be built into how the work gets measured.

The pope aims these principles outward, at states and the large platforms. The harder thing for product leaders is to aim them at yourself. It is easy to diagnose the broken system above you. It is harder to ask whether the team you run demands courage for things that should be routine, and then redesign that.

Borrowing the structure, not the faith

I am not borrowing the faith underneath any of this, and I am not turning a product argument into a religious one. The structural answer is centuries old, about how groups of people build things. The problem is real. Don’t build your city, or your org, on one person’s bravery. Spread the responsibility, hold it together, and shape the conditions so doing the right thing doesn’t take a hero.

The nuns who drilled structure into me would probably find it funny that an encyclical is what made the idea click. Then again, maybe they wouldn’t be surprised at all.

Where in your organization does doing the right thing still depend on one person being brave enough to do it?

Photo: “Jalen Brunson” by Erik Drost, licensed under CC BY 2.0.

But the most important thing Brunson did off the court was a contract. A year earlier, he signed a four-year, $156.5 million extension instead of the five-year deal worth roughly $269 million he was eligible for. He left about $113 million on the table. That decision kept the Knicks under the league’s spending limits and gave them the room to hold their roster together and keep adding talent.

The defining move of the team’s best player was not heroic. It was structural. He used his leverage to fund the team around him.

That is the part that product leaders should consider.

Leadership is the system you build, not the output you produce

For a while, I have been thinking about the dimensions of product leadership and how unevenly they show up across organizations. Marty Cagan talks about the responsibility of product leaders to provide strategic context and coaching. Shreyas Doshi describes archetypes that highlight where leaders are naturally strong and where they are not. Both resonate because they reflect what I see every day. Product leadership is multi-dimensional, and the gaps are predictable.

I have always been strongest in discovery & definition: identifying and framing the problem, shaping the solution, and driving delivery. I lead with empathy by supporting the team, respecting boundaries, and helping people grow. The pillar I have had to strengthen consciously is the technical one, understanding how we build, how systems fit together, and what the real constraints are. I have not coded hands-on in decades, but I have always known enough to ask the right questions, and I keep pushing that further.

Most leaders are like this. Some are deeply technical but light on product sense. Some are strong in delivery but have never spent real time in discovery. Some set exceptional strategic context, and others assume the team will figure it out. None of that is unusual. Our instincts come from where we ‘grew up’ professionally.

The instinct that hurts teams is the one we tend to admire most: being the person who personally closes every gap. Call it heroics; in reality, it’s one individual compensating for whatever the system does not provide.

The hero move and the structural move

Watch any struggling team, and you will often find one person holding it together through sheer effort. The leader everyone escalates to. The one who personally rescues the launch when it slips. The one whose week away stalls every decision. From the outside, it looks like commitment, and it is easy to mistake for strength.

But individual heroics are a signal, not a solution. They usually mean the strategy is unclear, the system is weak and lacks trust, or the talent around the leader is too thin to carry the load. The hero is compensating for a structure that was never built.

Brunson is the inversion of that pattern. The most visible player on the team made his most consequential move off the court, in a contract he signed a year earlier. By taking less money, he gave the team room to build depth around him.

None of this is an argument against courage. Brunson’s calculated choice took real courage. It was costly and personal, and he made it on purpose. In his words, “I want this team to be together for a long time.” That is the line between heroics and courage. Heroics is a reaction when a system is already failing. Courage is a deliberate choice you make before the moment forces your hand.

The pillars of product leadership reinforce or weaken each other in the same way:

• Strategy without empathy becomes brittle.

• Discovery without technical depth becomes fantasy.

• Delivery without product sense becomes project management.

• Coaching without strategic clarity becomes noise.

Strong product organizations do not rely on heroics, they build the structure that makes heroics unnecessary:

• Creating strategic clarity so people understand the problems, the bets, and the constraints.

• Raising talent density through hiring, coaching, and making the hard calls.

• Building a shared language: problem to hypothesis to metric, outcomes over outputs.

• Designing the system, not just the skills. Incentives, team structure, autonomy, and culture matter as much as individual competence.

For Brunson, raising talent density was not an abstraction. He paid for it out of his own pocket.

Why this matters more now

AI is sharpening the point. The individual output that used to make a product leader look indispensable is exactly what AI now drafts in minutes: the strategy memo, the PRD, the research synthesis, the board-ready deck. As that work commoditizes, what remains is the work you cannot automate: the judgment to design the system, raise the talent around you, and set the principles a team aligns to. AI is a lever, not a replacement for accountability. The edge shifts from the output you can produce to the structure you can build.

So I will ask my fellow product leaders the question I am asking myself: which of your “heroics” is actually a sign that your system is not built yet?

‌This reflection is part of an ongoing series on building a product culture: how teams think, decide, and build products.

For a while, the product work felt the way it was supposed to.

Then that leader left. Maybe it was a better opportunity, a reorg, or maybe a layoff.

That leader was holding the whole thing together with duct tape and glue, and it held right up until they let go. We praise leaders like that, and we should.

The person who replaced them wasn’t a villain. They just played a different game, maybe the one the rest of the company rewards. Perhaps it was the deficit of trust or a different leadership style that you lost your autonomy in product decision-making. The team went back to measuring outputs (what we were shipping), instead of outcomes (what problems we were solving). A few cycles later, you reverted to being a feature factory, not building products.

A team that only works because of one specific leader isn’t a system. It’s a personality trait, and personality traits leave when the person does.

I’ve named the alternative Structural Courage, and I’ve written about it in pieces. The short version: Structural Courage is when the system takes care of the things we normally need individual bravery for. Telling the truth, killing an experiment that’s secretly your baby, saying out loud that you got it wrong. On most teams, those take a deep breath and a good day. The point is to make them ordinary.

Since writing and speaking about it, I have been thinking about what it takes to build Structural Courage. So let me be specific. There are two pieces you can put in place on your own team - Enabling Constraints and Operational Autonomy, and a third that decides whether any of it outlives you - that’s density. Let me explain.

Enabling constraints. A constraint is just a boundary, but the right one makes the behavior you want the path of least resistance. Without it, people drift back to whatever the system rewards. I learned this the slow way. I taught my team to ask “what problem are we solving,” but the demos and planning asked “what are we delivering”, and within weeks, they were listing features.

So, I rewrote the performance rubric for how I measured the team. Instead of asking for a list of which features you shipped, it asked which user problem you solved and what it did for the customers and the business. You can’t answer that with a backlog. Most of my team couldn’t answer it at all at first, because they’d never actually measured impact, so they had to go build the instrumentation for it. The constraint forced new work into existence.

Operational autonomy. This helps push decision-making down to the teams that have the information, without coming to ask first. I got the mechanics from David Marquet, the submarine captain who took over the worst-performing ship in the fleet and learned to stop giving orders. His crew started saying “I intend to” instead of waiting to be told. But the crucial part is that he didn’t just hand over the keys and hoped for the best. He built two things first - Competence and Clarity. The team had to be competent to know what was actually safe (do they know how?), and they had to have clarity about why it mattered (do they know why?). Competence and clarity work together. Clarity tells you what matters. Competence gives you the confidence to defend it.

Density. Here’s what I got wrong doing all of this. I built enabling constraints and operational autonomy on my team. And it worked until I left. I had built something that still needed me to sustain it, which is a hard thing to admit about work you were proud of. The research on how norms spread says you need somewhere around a quarter of a group behaving the same way before the behavior spreads and stops needing a champion. I didn’t build density, I was the density. One brave team is not a system. This only survives when enough people carry the behavior that no single one of them is exposed for holding it.

If you take one thing from this, watch the questions that get asked in your next planning meeting - when will something be delivered, or why it matters. That’s the system telling you what it actually rewards. If the questions don’t change, the system won’t either.

Structural courage is building that foundation on purpose, so the next time a good leader moves on, the team they shaped keeps working without them.

Don’t be the hero. Build the system so you don’t need one.

I wrote about how to tell the story of the product strategy before. That playbook hasn't changed much. What's changed is that the time saved on documenting doesn't compound to the time needed for agreement and alignment.

On one strategy I led, I made an 18-minute podcast using NotebookLM, 3-minute video reels in Canva, and circulated the whitepaper everywhere. I was on a roadshow with every leader I could find. The artifacts scaled. Unfortunately, the alignment didn't.

Once I shared the strategy with my most important stakeholder, a known dissenter, and assumed that one presentation had us both agreed and aligned. She was neither. That backfired very quickly when she called me out on it publicly. I learned the lesson to not assume, but do better stakeholder management by being available to answer all questions that come out of the discussion, repeatedly.

A comprehensive strategy with priority bets and non-priorities is just the start. Getting it adopted by your team, your stakeholders, and your leaders takes weeks, if not months. Constant conversations, repeating the strategy until everyone can answer what the product does, what problem it solves, for whom, how it works, and which bets you're placing to find out.

When done well, agreement happens in the meeting. Alignment is when your stakeholder and team can articulate your strategy in their own words, consistently, even when you're not in the room.

The test I use: is your stakeholder the advocate for your strategy and can walk through it, when you're not in the room? If they can't, you haven't aligned.

AI helps you produce the artifacts for more touch points. Podcasts, summaries, recap reels, FAQs etc. It cannot replace the conversation.

What's the longest you've spent getting a strategy adopted, compared to how long you spent writing it?

Documenting a strategy has been compressed. The artifact, the deck, the one-pager, and the proposals can be generated faster with AI than ever before.

But the thinking doesn’t compress. Deciding what you’re betting on, who your product is for, what you won’t do, and how you’ll know it’s working. That still takes hours of staring at a wall of AI-generated text. AI can create the document in 20 minutes. It cannot tell you what your strategy is. AI can research your competition, but it cannot decide which bets are worth placing. AI can synthesize the experimentation data, but it cannot prioritize what comes next.

This may be a contrarian opinion but I think that we need to define comprehensive product strategies that clearly articulate the product, user segments, competitive advantage and open questions we haven’t answered.

Strategy is as much what we would do as it is what we wouldn’t do. The non-priorities are essential to be included in the strategy document to bring clarity to the team and remove the guesswork of whether something was essential to execute the strategy.

Non-Priorities could be things that we are never going to do, and the things we are not doing right now. This makes the priorities clear to the team and helps bring focus.

A clear strategy also helps defend against the noise, your loudest stakeholders, and a wish list that someone has. When a new request comes in, a detailed, clearly articulated strategy can help anchor the team and help make a case for no or not-yet.

Strategy is also the spine that holds team decisions together when the cost of building collapses. When everyone can ship faster, every team has a plausible reason to plant a flag. Without explicit non-priorities, the strategy is silent on what to refuse, and capacity goes to whoever asked loudest or moved fastest.

This is the upstream version of the agent land grab I wrote about. When a team can’t name what it isn’t doing, every new request, including the new agentic one, gets a yes by default. The “compared to what?” question presumes there is a clear strategy to compare against.

The question for your next strategy review isn’t whether the document is short enough to read in 20 minutes or can be created in 20 minutes using AI. It’s whether the non-priorities listed are honest, clear, and consistent with the priorities.

What is your team explicitly not doing this quarter, and what does the rest of the org know about it?

I see roadmaps where teams have an agentic architecture, and almost none of them start with the customer. The questions being debated are who owns it, who builds it, and who gets credit. The AI agent becomes a power vector before it becomes a product.

When ownership fights break out before the user's need is defined, we end up with an agent that protects the ego (and roadmap) rather than solving a problem. The team that planted the flag first has a line item to defend. The team that didn’t is racing to plant the next one.

The way I see it, two land grabs are happening at once. One where the consulting vendors and AI labs are racing to own the agentic layer, and every demo is making the same claims. The other is the internal one inside companies, where teams are competing to own their ‘agent for x’ before another team does. Product, platform, engineering, and the AI function are all reaching for the same territory.

Each of those teams has a plausible-sounding claim. The platform team owns the shared services and APIs, so the agents built on top look like theirs to claim. The data team owns the data, so the agents that touch it look like theirs, too. The AI function owns the model layer and orchestration tooling, so the agents that depend on them look like theirs to direct. None of that is the same as owning the customer’s experience. Don’t get me wrong, platform agents have a place in the world. They just have to serve upstream user needs, not define them.

The concerns of ownership, domain expertise, and customer experience impact belong at the start of these conversations, but get raised after the architecture has already been chosen, if they get raised at all.

Anyone can build an agent now. I have seen several convincing demos because the capabilities, the infrastructure, and the tools are mature enough. The race has shifted from who can build the thing to who gets to. Who has the budget, the team, and the airtime in the next leadership meeting. The hard work is still the orchestration and judgment about what to build.

The external vendor war is running the same race on a bigger scale. Microsoft is positioning Agent 365 as the governance layer for everyone’s agents. Google, OpenAI, and Anthropic are embedding dedicated forward-deployed engineers inside enterprises. Every demo from a vendor or competitor becomes an argument for “we need our own version of that, and my team should own it.” That’s how a strategic question, ‘should this be an agent at all?’ never gets asked. It gets replaced with, ‘who runs the team that builds it?’

There is a cost of not starting with the customer problem. When agents are built at different layers, and the output is wrong, who owns the evaluation, the fix, and the long-term sustainment? We are not thinking about that now because we are so busy land-grabbing the build, not considering whether we will have the capabilities or the team to support it long term.

This is what the ownership land grab costs. We picked a stack because someone wanted to own it. We shipped on top of it. Nobody is wired up to catch the failure when it happens. The customer sees one product. Internally, three teams point fingers at each other.

That’s not hypothetical. Deloitte’s 2026 State of AI in the Enterprise survey of 3,235 leaders found that only 21% of organizations have mature agentic governance. The median enterprise is shipping agents into production without the operational accountability to handle failures.

Arguably, a cross-functional operational role that owns the agent’s behavior in production, including its evaluations, its observability, and its divergent paths, could solve the fragility problem. The product manager could absorb these responsibilities. But this is still downstream of the decisions that have already been made based on politics.

Ownership must be defined before the agents are built. The human accountable for the decision to build this agent in the first place, who can answer what it replaces, what the delta is, what incremental value it delivers, and for whom. Industry research suggests that around 56% of enterprises that successfully scale agents have a named owner from day one. That’s not a process artifact. It’s a forcing function. You can’t name the owner without first deciding what the agent is for, what it replaces, and what success looks like. These questions get preempted when the ownership decision happens first. You can’t measure incremental value once “whose team builds it” has already been answered.

And sometimes the honest answer to “should this be an agent?” is no. The right answer could be an API call or the LLM workflow you already have, made faster. The PM job is knowing the difference. That’s the question worth bringing back to every roadmap conversation. Is this an agent because we need one, or because someone wanted to own one?

Anyone can ship an agent this week. The capability is no longer a scarce resource. What’s left is judgment about what to build, what not to build, and who is accountable when the stitched-architecture fails.

That’s still the PM work.

When agents came into the picture, Google came up with protocols (A2A) for agents to find and interact with each other. Marketplaces have launched so agents can be discovered. A whole genre of work is forming around agent orchestration patterns.

An AI agent is a single LLM-driven system that performs a defined task using tools. An agentic system layers architecture on top: an orchestrator agent decides which sub-agents to invoke, stitches their outputs into a result, and adapts based on context and instructions.

All of this is reshaping how teams work, and I have one question I haven’t seen many people ask out loud or answer yet. Gartner predicts more than 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. Product leaders are approving migrations to agents faster than we’ve set the bar to evaluate them.

There’s a pattern playing out everywhere right now. A team ships an agent that resolves a specific user or business problem. The proof of concept is convincing, the demo runs clean, executives are supportive, and the project gets the green light to scale. No one asked what the existing workflow was already resolving or at what cost. Even teams that do measure value typically measure it on the new system alone, not against the workflow it replaced. Value delivered is not the same as incremental value delivered.

Consider a specific case. Migrating an existing conversational AI experience from LLM responses to an agentic architecture means rebuilding the workflow end-to-end and redoing instrumentation. To be worth it, the new architecture has to deliver incremental value over what the LLM workflow already does. Would you rather improve LLM response accuracy and latency on the system you already have, or rebuild it on a new architecture for an unproven delta?

Agentic systems cost more to operate than workflows do in terms of evaluation, observability, escalation paths, and retraining. If the delta over the workflow is zero, you’ve imported cost without delivering incremental value.

So here’s my challenge to product leaders: if you already have a workflow, does moving to an agentic architecture offer incrementality? Could you reach the same outcome by staying on an LLM-based workflow? What does the system deliver above and beyond your existing workflow, now, in a year, in five years? Will agentic systems even be part of the architecture two years from now, or will they consolidate into something else?

Before approving any workflow-to-agent migration, ask one question: compared to what? Every well-built system delivers value. What matters is the delta over the system you already have. If you cannot name the workflow being replaced and write down the delta you expect, you are running a land grab dressed as a migration.

I asked the first PM presenter what the benefit of their feature was. They paused, then asked whether this was a feature demo or a call to share success metrics. They weren’t tracking the benefit. The business had set the targets.

As more PMs presented, only a few came prepared, and more of them started mentioning metrics. They’d seen what got asked earlier and adjusted. After the call, some sent me their data. A few pinged me to discuss how we ensure these questions are part of the format.

The right questions were alive in DMs. They were dead in the meeting room.

This is performance theater. We pattern-match to what’s being graded, not to what the work asks.

We live in a culture where we hand out ‘good jobs’ for the bare minimum. We give participation trophies so no one feels left out.

It shows up across PM work:

• Demos that celebrate launch dates, not outcomes

• Roadmaps that track features, not bets

• Status updates that report activity, not learning

AI raises the velocity, not the bar. Shipping faster with AI is still just shipping if we’re not measuring the impact and outcomes. When iteration gets cheaper, judgment matters more.

Showing the feature we shipped is the bare minimum. What we get paid for, sometimes very, is insight and judgment.

When we’re doing the job, we don’t wait for success to be reported to us. We look at our data, know the number, and answer “compared to what, by how much, and what did we learn.” We ask those questions in the room, not in the DMs afterwards.

This is what makes us PMs and not feature managers. AI is already taking the second job.

I have run a version of this exercise multiple times, and each time a surprising thing happens. No one walks over to the other group to discuss the dependency that is obviously mentioned in the strategy. I have been thinking about why that is happening.

We see this everywhere in our daily work dynamics. We are consumed by the ‘us vs. them’ mentality that we don’t notice the system we are operating in. Sometimes what we do, how we behave, and the questions we ask are exactly what the system is designed for. In an unintentional system, people see what gets rewarded and behave that way.

In the workshop, no one walked over because I designed an exercise that rewarded staying within the group. Each team owned a strategy. Each team got points for finishing the template, defining bets, setting metrics, and navigating their stakeholder. The system had no explicit instructions to discuss the dependencies with the other team. Crossing over wasn’t asked for and wasn’t rewarded, so rational participants stayed in.

This goes back to my thesis on enabling constraint, that our system design intentionally or not invokes a desirable or undesirable behavior. The workshop exercise simulates the system most teams operate in, where the coordination prompts are not explicitly defined. Teams discuss these in stand-ups and planning ceremonies.

We need to create an ‘empowered’ system where these prompts are not needed, and the system makes it obvious with clear contracts, accountability, and an environment that encourages the right behavior.

Next time I run the exercise, I’ll add a coordination prompt. The default behavior in any system tells you what the system is optimizing for.

What’s your system designed to do?

At Universal, I checked the app a few times a day to glance at wait times. Are the lines short enough to ride again? At Disney, I refreshed the app all day, trying to catch Lightning Lane availability before it disappeared. Lightning Lane availability pops up at unpredictable times throughout the day. If I missed the window, we missed the ride.

One park had us present with our kids, and the other had us managing a system.

The gap between a system that works for you and a system that asks you to work for it is a product design lesson that felt so obvious, I couldn’t ignore it.

Where we stayed changed everything else

We stayed at the Portofino Bay, which is one of Universal’s on-site resort hotels. That one decision of where to stay shaped our entire trip. Universal’s Express Unlimited pass was included with the room. It’s usually an upsell if you’re staying at a non-Universal property. Once you have the pass, there is no additional cognitive overhead of tier selection or budget math each morning. It was just there.

The boat ride to the parks from the hotel ran every few minutes. Security screening happened at the dock before we boarded, not at the park entrance with thousands of other guests. By the time we stepped off the boat, we walked straight to the park entrance.

At the turnstiles, Universal uses facial recognition for “photo validation.” We scanned our cards once each day to link our faces. After that, we just walked up to the express lanes. Our faces were the credentials, so no fumbling for phones, no scanning wristbands, no pulling up the app was required.

The pass, the transport, the security, the entry — all of it managed the complexity before we ever encountered it. Nobody at Universal had to rescue the experience for us. The system never created a moment that required a person to step in and explain something or fix something.

How often can we say that about the products we build? How often does one good upstream decision make everything downstream easier for our users?

The same ride, three times

Express Unlimited didn’t just save us time. It changed how we experienced the parks.

My kids are really into Harry Potter right now, so the rides were the ones we experienced the most. The first time through, they felt the thrill. The second time, they started catching story details, the portraits moving, the spells, and the characters. By the third ride, they were pointing things out to each other, fully absorbed in the world.

That only happened because Express Unlimited doesn’t limit you to one ride per attraction. We didn’t have to plan which rides to prioritize; we just followed whatever our kids were excited about.

Disney’s Lightning Lane works differently. One ride, one redemption. If you want to ride again, you’re back in standby or selecting/buying another pass, if it is available. It steers you toward breadth, so you see as many things as possible, rather than letting you go deep on the ones you love.

And it doesn’t just limit what you ride. It limits how your family moves through the day. One evening at Hollywood Studios, it was raining, and our Slinky Dog Dash Lightning Lane was booked for 7:30 pm. So instead of heading back to the hotel, we waited around in the rain because we’d lose the reservation if we left. The system was managing our evening, not us.

The way I see it, that’s a real design difference. Universal is built for the family that finds something they love and wants more of it. Disney built for the guest who needs to be moved through a sequence. And every time a system moves people through a sequence instead of letting them self-direct, it creates moments where someone has to pick up the slack. Those decisions come from the system, not from the people in it.

Disney knows how to remove friction

As a product person, this is what made the trip so interesting to me: Disney is clearly capable of designing for the guest. Their Imagineers are some of the most studied experience designers in the world.

The app’s integration with mobile food ordering works really well — pick a restaurant, order from the app, and show up when it’s ready. MagicBand handles payments, hotel room entry, and park access in one tap. Dining reservations are fully integrated into the app.

Disney knows how to make friction disappear. So when Lightning Lane is confusing, that’s not a capability gap. It’s a design choice.

And the numbers back it up. The predecessor to the Lightning Lane was Disney’s FastPass+ system, and it used to be free. It was included with park admission from 2013 until it was quietly discontinued during the COVID closure in 2020. It never came back.

What replaced it was Genie+, which launched in October 2021 at $15 per person per day. That became Lightning Lane Multi Pass, which now costs $24 to $45 per person per day, depending on the park and date. If you want to skip the line for a premium ride like Guardians of the Galaxy or Tron, you have to buy a separate Lightning Lane Single Pass — $7 to $25 per ride, per person. If you want everything included, then the Lightning Lane Premier Pass runs up to $449 per person for a single day at Magic Kingdom.

Disney’s leadership has been open about the strategy. On a 2022 earnings call, then-CEO Bob Chapek described the parks as playing a “yield game” — using dynamic pricing and reservation systems to “increase per-capita spending meaningfully.” Per-guest spending was up 40% compared to 2019. His successor called the pricing “alienating” — but kept the system in place.

What used to be included became a revenue layer. The tiers, the time windows, the daily purchase decisions, the pop-up availability you have to catch — that’s where the yield lives. And as a parent standing in the park with two kids, every moment of confusion that system creates is a moment someone has to deal with. Either I figure it out, or the person working the terminal does.

Who carries the friction?

Both parks had kind, pleasant staff. Nobody was rude to us at Universal. Nobody was unhelpful at Disney. The people at both parks were nice.

What was different was what each system asked of the people in it.

At Disney, the cast members are warm, well-trained, and helpful. They have to be — because the system keeps creating moments that need human recovery. A family whose Lightning Lane expired because a ride was closed during the window or while they were figuring out where to eat. A guest who bought a Multi Pass but didn’t realize the ride they wanted required a separate Single Pass purchase. A parent trying to understand why the pop-up availability they saw ten minutes ago is already gone.

None of those moments is the cast member’s fault. They’re the system’s. But the cast member is the one dealing with them a hundred times a day.

I see this in product work all the time. We offer 8 plan options with dynamic pricing, and sales reps have to untangle them for customers. We add complexity to the product and then add documentation to explain the complexity. The friction doesn’t go away — it just shifts to the people using the system and those operating it.

The way I see it, Universal didn’t build a system that requires its people to be exceptional. It built a system that lets them be human.

What are we asking of our people?

I came back from this trip thinking less about theme parks and more about the products and teams I’ve worked inside. At least Disney’s complexity is a deliberate revenue strategy. Most of our organizations don’t even benefit from the complexity we’ve created. We just live with it, ask people to work around it, and sometimes the system is generating the very problems we’re hiring people to solve.

I keep coming back to whether we’ve done the work of designing systems that don’t waste our team’s talents.

Universal didn’t get everything right. Their app is clunky and less integrated than Disney’s. The digital experience has real gaps. But they got the system right in the places that mattered most. We felt it in every ride, every entry, every moment we were with our kids instead of managing a product.

I’ve written before about structural courage — the idea that a well-designed system removes the need for individual heroism. That people shouldn’t have to be shock absorbers because the system should absorb it for them. I wrote that about organizations and teams. I didn’t expect to feel it so clearly on a family vacation. But walking through Universal with my kids, that’s exactly what it was. A system that had done the hard work so the people inside it didn’t have to.

That’s what good system design feels like. The people were just people — friendly, doing their jobs, not rescuing anyone from a broken process. And that was enough.

Outcome vs. impact

Where the outcome is the change in user behavior, impact examines whether the change mattered.

We changed the checkout flow, and users completed the purchase faster (behavior changed). Did the revenue go up? Did the support calls go down? Did customers come back?

Teams can drive behavior change that doesn’t produce value. More clicks, more time on page, more logins — none of that is impactful unless it connects to something that matters for the user, for the business, or both.

Benefit vs. impact

Benefits are the upside we expect. Impact is what actually happened, including the things we didn’t plan for.

Benefit is direct, intentional, and always positive. It’s the short-to-medium term upside we planned for.

Impact is wider, often systemic, and can be positive, negative, or neutral. It’s the longer-term effect of the change, planned or unplanned.

A feature that increases engagement but burns out the support team has a positive benefit on paper and a negative impact in practice.

Benefits are the result of the bets we placed and the value we delivered; impact is the effect of the change.

Why this matters

Teams that only measure benefits stop too early. We ship, the north star metric turned green, and we move on. Leaders who only ask for benefits get a filtered view of what the team intended, not what the product caused.

When we skip the impact question, we create a blind spot where negative consequences go unexamined. The checkout flow is faster, but did returns increase because users are buying impulsively? The platform migration sped up one team, but did it break three others?

These are uncomfortable questions. That’s the point. Benefits tell us what we hoped would happen. Impact tells us the full story.

In the f(Clarity) framework, purpose examines why, users look at who, and impact tells us whether any of it mattered.

If we stop at the benefit and don’t look at whether it mattered, we are only telling half the story. What’s something your team shipped recently where the benefit was clear but the impact wasn’t?

The charitable read is that writing good evals requires the same rigor as a good PRD — clear success criteria, edge cases, and user intent. If that’s the argument, I’m with it. Good evals are hard to write, and they force the same precision we should have been bringing to PRDs all along.

But that’s not what was said. The claim was that evals replace PRDs. And that’s where the framing falls apart.

These are not the same cognitive job

Evals measure model behavior against defined criteria. PRDs define the problem worth solving, who it’s for, and what success looks like for the user. Conflating them doesn’t elevate evals; it just misunderstands PRDs.

An eval can tell us whether the model’s response was accurate, concise, and free of hallucinations. It cannot tell us whether we’re solving a problem worth solving. It cannot tell us who the user is and what their struggling moment looks like. It cannot tell us whether the product should exist at all.

Last week, I wrote about product mindset as a function of clarity — clarity of purpose, users, and impact. Evals measure whether the thing we built is working. But they can’t do the upstream work of defining purpose or identifying users. If any of those are missing, it doesn’t matter how good the eval suite is; the product may still lack purpose or solve a problem for the wrong users.

I’ve built evals. They’re not the starting line.

We built a rubric to evaluate every conversation customers had with our chatbot — measuring accuracy, relevance, conciseness, hallucinations, and toxicity. As LLMs evolved, we updated the rubric based on how customers were interacting with the product, and then worked on automating the evaluation to review conversations at scale.

That eval work was valuable. But we could only write those evals because we had already done the harder work — defining what the chatbot was for, who it served, and what a good experience looked like for those users. The eval didn’t replace that thinking. It depended on it.

The real problem with “X is dead” framing

This is the same type of rhetoric that says PRDs are dead because you can show (not tell) your idea in Lovable. Or that product management itself is dead. These titles make good clickbait. But what are they telling the product management community? Your area of expertise is dead. Your artifacts are no longer needed.

AI has changed how we work and the artifacts we produce. We no longer need to spend hours on manual documentation. But the judgment underneath — what to build, for whom, and why — hasn’t been automated. The decisions we make from empathy for our customers, understanding of our cross-functional teams, and relationships with stakeholders are the PM job that will remain human. The documents were never the point; they were the output.

Evals are powerful. They’re not the starting line.

Evals belong in every AI PM’s toolkit. That’s how we validate that the product is doing what it’s supposed to do. But they sit downstream of the clarity work — the purpose, the users, the definition of impact. Just like documents, evals are output. The thinking is still the job. Without that, we’re measuring precisely and building aimlessly.