“Before you use those figures, speak to Janice.”
Most people who have worked inside an organisation will recognise that sentence. The finance dashboard is technically correct, but Janice knows the data is delayed by 14 days, that two product lines are excluded, that refunds are temporarily being misclassified and that the latest acquisition has not yet been integrated into the reporting. None of that appears on the dashboard.
An experienced product manager knows to speak to Janice first. They know which data can be trusted, which needs qualifying and which should not be used at all — an understanding accumulated through meetings, mistakes, relationships and time spent inside the organisation.
Now imagine that same PM asks an AI assistant to analyse the figures, compare investment options and recommend where the organisation should focus. Unless Janice’s qualifications have been captured somewhere the AI can access, it will not know them. It will produce a clear, confident and thoroughly reasoned recommendation based on an incomplete interpretation of reality.
The problem is not bad data. It is the distance between what the organisation’s systems formally say and what its people know must be taken into account.
Call it caveat debt.
As product managers increasingly use AI to analyse evidence, challenge priorities and prepare decisions, caveat debt is one part of a larger risk: the emergence of reasoning silos.
The bottleneck has moved
In my previous article, Everyone’s Building Faster. Now What?, I argued that as AI reduces the time and effort required to execute work, the relative value of judgment increases.[1] When every organisation can research, design, analyse and build faster, speed stops differentiating. The important questions move upstream: what should we do, which opportunities matter most, what evidence should we trust, what should we stop doing?
The quality of those decisions depends on the information beneath them — strategy, objectives, customer evidence, commercial realities, operational constraints and previous learning. That information has always been messier than organisations admit. Humans compensate by remembering conversations, recognising outdated documents and knowing whom to ask before trusting a particular figure.
AI has no such organisational instinct. It can only work with the context it can access and the qualifications made visible to it. That makes coherent, current and evidence-based organisational information more valuable than ever.
But good information is only half the challenge. Product leaders also need to consider what happens to the reasoning performed against that information.
What is a reasoning silo?
Your product managers are already experimenting with AI — synthesising customer research, interrogating performance data, comparing opportunities, challenging roadmap assumptions, preparing investment cases, drafting requirements and stakeholder communications.
Encourage this. Everyone is still learning what these tools can do, where they fail and how to get useful results from them.
Some PMs will become exceptionally good at it. They will build carefully curated context libraries, develop multi-stage procedures and agents, and learn how to challenge an answer, test its sources and recognise when a model is producing something plausible but unreliable. Those people are not the problem — they may be discovering the practices that will define AI-enabled product management in your organisation.
The risk appears when their context, methods and reasoning remain personal.
A reasoning silo forms when the information, procedures and deliberation behind a product decision are private or person-dependent — even when the resulting output is shared. The organisation receives the roadmap, recommendation or business case. It does not receive the sources the PM selected, the qualifications applied to them, the assumptions given to the AI, the alternatives considered, the model errors corrected or the contrary evidence rejected.
The output is visible. The capability behind it is not.
It is worth separating two things that behave differently.
Method is portable. The way a PM interrogates an opportunity, the questions they insist on answering before writing a requirement, the checks they run against a synthesis — these can be written down, taught and reused. They may possibly stay true for years.
Context is perishable. Janice’s qualification is a fact about one dashboard in one quarter. Write it into a shared playbook and it will be wrong by spring, quietly, in the same way the dashboard was wrong.
Method can be written down. Context has to be maintained. The distinction matters because the remedies differ. Method silos close through documentation and shared practice. Caveat debt does not. A caveat captured once and never revisited becomes false assurance — worse than no caveat at all, because now nobody thinks to ask Janice if anything changed.
Product reasoning was already fragmented
AI did not create this problem. Most product organisations already struggle to connect strategy, prioritisation, roadmapping, discovery, delivery and learning. Different teams prioritise from different evidence. Roadmaps contain commitments whose real origins are undocumented. Customer research sits in separate repositories, commercial promises live in CRM systems or account managers’ memories, and decisions made in leadership meetings do not always find their way back into the strategy documents teams are using.
In practice, many organisations do not have one shared product decision system. They have multiple local systems connected through meetings, presentations and personal relationships. Experienced people make this work by carrying context between them. They know that the “strategic priority” is also linked to an important customer renewal, that an apparently attractive market has regulatory complications, that a manufacturing lead time makes the proposed launch date unrealistic.
That tacit knowledge has always made the organisation more dependent on individuals than its formal operating model suggests. AI intensifies the dependency — because instead of repairing the shared context, each PM can now assemble their own version of it.
How reasoning silos develop
The pattern is predictable. A PM asks the approved organisational AI assistant a question about product priorities and receives a generic answer, because the shared information is incomplete, stale or difficult to retrieve. So they build a personal context collection: their own notes, preferred strategy documents, customer interviews, local analytics and meeting summaries. The answers improve. Over time they refine their prompts, add sources and develop a reliable process for challenging opportunities and preparing recommendations. More of their most valuable thinking happens inside this private environment.
The final outputs are copied into the roadmap or a presentation. The reasoning process, corrections and accumulated context are not written back. The shared capability stays weak, so the next PM follows the same path and builds another private collection.
Picture the aftermath. A PM moves on, and their successor inherits a roadmap full of well-argued commitments — but the personal workspace of prompts, context files and corrections that produced them has left the building. The documents remain; the reasoning does not.
The cycle reinforces itself: Weak shared context leads to private context. Private context produces better local results. Better local results encourage more private reasoning. Private reasoning does little to improve the shared context.
The organisation becomes faster without becoming more intelligent.
This is not hypothetical. Bottom-up adoption is already a defining feature of workplace AI: Microsoft and LinkedIn reported in 2024 that 78% of surveyed AI users were bringing their own AI tools to work.[2] A 2025 study of 885 product managers at Microsoft found extensive individual experimentation, alongside concerns about accuracy, workflow integration and accountability.[3] The exact practices differ by organisation and industry. The dynamic is the same: individual experimentation develops more naturally than shared organisational capability.
Faster work can mean less collective thinking
There is a subtler risk. AI enables PMs to complete work that previously required interaction with researchers, analysts, engineers, finance, operations, legal or customer-facing teams. Some of those interactions are unnecessary handoffs, and removing them is pure gain. But not all collaboration is administrative overhead.
A conversation with finance does not simply transfer a number — it exposes the qualifications attached to that number. A conversation with an engineer does not simply produce an estimate — it may reveal that the team has framed the problem incorrectly. A conversation with customer service adds human context about which problems are frequent, which are emotionally significant and which customers have learned to work around.
These interactions are where the organisation makes sense of incomplete information. They introduce disagreement, interpretation and perspectives that are not present in the source material, and they create shared ownership of the eventual decision. AI allows a PM to produce a highly polished, locally coherent recommendation without encountering any of that challenge — more analysis, less time, fewer perspectives.
So when AI removes a handoff, ask: Did it eliminate delay — or eliminate a perspective?
Automate the transfer of information. Be far more careful about automating its collective interpretation.
The superstar paradox
The people creating reasoning silos may be producing the best AI-assisted work in your organisation. They have taken the initiative, learned through experimentation and invested in building context, procedures and judgment. Find these people. Do not restrain them.
But personal excellence is not organisational capability. When an effective AI practice stays with one person, other PMs cannot reproduce it, its quality cannot be evaluated consistently, central processes do not improve, and the capability disappears when that person changes roles or leaves. The organisation retains the individual’s final documents but loses the system that produced them.
Closing that gap does not mean centralising every prompt or forcing everyone to work the same way. It means creating a mechanism through which valuable local discoveries become shared organisational assets.
Explore locally, learn collectively
The wrong response to reasoning silos is to stamp out individual experimentation. The other wrong response is to choose one tool, publish a central prompt library and declare that the organisation now has an AI operating model.
The better approach: Explore locally. Learn collectively. Standardise selectively.
Let PMs keep trying new tools and methods within appropriate security and data controls. Then create ways to identify practices that are producing genuinely better results — not simply faster or more polished outputs. Study what effective practitioners are doing: what context they assemble, how they verify the output, where human judgment enters, which parts work only because of their personal expertise and which could help other teams.
Treat strong local AI practice as a form of organisational research and development. Some experiments will remain personal productivity techniques. Others should become shared playbooks, common context services, training or embedded parts of the product operating model. The job is not to make every PM work identically. It is to ensure the organisation recognises and retains valuable capability when it appears.
Build a trusted context layer
Shared AI capability requires more than gathering documents into one repository. Product context comes from documents, databases, APIs, customer conversations, research systems and operational platforms. For that context to be trusted, people and AI need to know where it came from, who owns it, when it was last updated, what it excludes, where it is known to be unreliable and which decisions it is suitable for – all the caveats.
The qualification attached to the information is as important as the information itself.
Forget the perfect “single source of truth.” The realistic ambition is an inspectable organisational context layer in which provenance, freshness, limitations and ownership are visible. The AI needs access not only to the finance figures, but to the equivalent of Janice saying: “These are suitable for directional planning, but do not use them for this quarter’s product-level profitability decisions.”
That is how caveat debt begins to be repaid.
Where reasoning silos bite hardest
The pattern shows up across the operating model, but three areas deserve particular attention.
Prioritisation. Different PMs ask AI to score opportunities using different evidence, prompts, assumptions and definitions of value. The resulting scores look objective and comparable while being based on different versions of reality.
Discovery. A PM uses AI to synthesise interviews and identify patterns without involving the researcher or service colleague who holds relevant customer context. The synthesis gets faster; the interpretation gets narrower and less open to challenge.
Outcome measurement. Recommendations are produced and approved without a clear record of the assumptions and expected outcomes behind them. When results arrive, nothing connects what happened to what the organisation originally believed.
The same logic applies to roadmapping, portfolio management and releasing: multiple polished but inconsistent local arguments, and no shared basis for enterprise-level trade-offs. In every case, local reasoning improves while shared decision-making and organisational learning stay weak.
Preserve collaboration where judgment matters
A stronger context layer does not remove the need for collaborative decisions. AI can prepare the evidence, locate contradictions and generate alternative interpretations — it can make a prioritisation meeting shorter and better informed. But consequential product choices still benefit from different functions interpreting the evidence together.
Decide deliberately where collaboration creates value. Not every document needs a committee, and not every analysis needs a meeting. But decisions involving significant investment, customer impact, operational change or risk should not become private conversations between one PM and a model. The operating model should preserve moments where assumptions are exposed, contrary evidence is invited, trade-offs are made explicitly, accountability remains human and the reasoning is captured.
There is a human consequence worth watching too. Product management is not only the production of artefacts. It is a collaborative process of making sense of customers, constraints and uncertainty with other people. If too much of that work becomes a private exchange between an individual and an AI system, the job becomes faster — and more isolated. A more efficient workflow is not always a better working environment, or a better decision system.
Make the organisation learn
The final test is whether AI-assisted work improves future decisions. A mature product organisation can connect:
Context → reasoning → decision → action → outcome → learning
Important decisions should leave enough of a trail for another person to understand what was believed at the time, which evidence mattered, what alternatives were rejected, who accepted the risk and what outcome was expected. After release, return to those assumptions and compare them with what actually happened.
This is where an AI-enabled product capability can become valuable: connecting present decisions to previous evidence, surfacing similar past choices and identifying assumptions that repeatedly fail. But only if the organisation captures the learning.
Enterprise research points the same way. Organisations reporting greater value from AI are more likely to redesign workflows, embed AI into business processes, involve senior leaders and create feedback mechanisms — not simply hand tools to individuals.[4]
A new responsibility for product leadership
Product leaders already design how work is prioritised, funded, roadmapped, governed and released. They must now also design how AI-assisted reasoning becomes visible, challengeable and reusable.
That does not require one central product brain dictating what every team should do. The better model is federated: local experimentation and individual working styles, built on shared organisational knowledge, common evidence standards, visible decision reasoning and deliberate collaborative challenge.
A few questions will reveal where your organisation stands:
- Where are your PMs doing their most important AI-assisted thinking?
- Are they working from the same strategic context and reality?
- Does the available context include known caveats and limitations?
- Can important recommendations be traced back to evidence?
- Do colleagues still have opportunities to challenge the interpretation?
- Can another PM reproduce the method, not just read the output? And would they know when the method is insufficient?
- What remains when an AI-enabled superstar moves to another team?
- Is each experiment improving only the individual — or also the product organisation?
The goal is not to stop PMs developing powerful personal ways of working. It is to make sure that when one PM discovers a better way to think, the organisation learns from it.
Experiment individually. Curate context collectively. Decide collaboratively. Learn institutionally.
Written by Eddie Pratt, Managing Director at Product Focus.
References
[1] Pratt, Eddie. “Everyone’s Building Faster. Now What?” https://www.productfocus.com/everyones-building-faster-now-what/
[2] Microsoft and LinkedIn. “AI at Work Is Here. Now Comes the Hard Part.” 2024 Work Trend Index Annual Report, 8 May 2024. https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part
The report was based on a survey of 31,000 people across 31 countries, LinkedIn labour-market data, Microsoft 365 productivity signals and research with Fortune 500 organisations. This article cites its finding that 78% of AI users were bringing their own AI tools to work.
[3] Ulloa, Mara; Butler, Jenna L.; Haniyur, Sankeerti; Miller, Courtney; Amos, Barrett; Sarkar, Advait; and Storey, Margaret-Anne. “Product Manager Practices for Delegating Work to Generative AI: ‘Accountability Must Not Be Delegated to Non-Human Actors.'” arXiv preprint arXiv:2510.02504, submitted 2 October 2025. https://arxiv.org/abs/2510.02504
The mixed-methods study surveyed 885 product managers at Microsoft, analysed telemetry for 731 of them and interviewed 15. As an arXiv preprint focused on product managers within a software company, it should be treated as emerging evidence rather than proof across every organisation or industry.
[4] Singla, Alex; Sukharevsky, Alexander; Yee, Lareina; Chui, Michael; and Hall, Bryce. “The State of AI: How Organizations Are Rewiring to Capture Value.” McKinsey & Company / QuantumBlack, 12 March 2025. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-how-organizations-are-rewiring-to-capture-value
The report draws on a global survey of 1,491 participants across 101 countries. It associates greater reported value from generative AI with practices including workflow redesign, senior-leadership involvement, embedded business processes, feedback mechanisms, training and defined adoption roadmaps.
Leave a comment