Why production agents burn compute on rediscovery, and what to do instead.
Up to 85% of AI agent compute is rediscovery. Re-fetching the same customer record. Re-reading the same doc. Re-asking the user questions the system already has answers to.
That number comes from Pine Cone, the vector database company, in launch material for a new product called Nexus shipped last month. Their pitch around it is sharp. Vector search alone is not the right retrieval contract for agents anymore.
That admission is the story. The bigger story is the wave behind it. SAP spent over a billion euros buying memory infrastructure. Google made knowledge architecture the headline at Cloud Next. Cloudflare shipped a memory product for agents. Microsoft keeps pushing graphs. Every serious infrastructure vendor is racing to fix the memory layer underneath production agents, and the move is happening in a single quarter.
If you are running agents anywhere in your GTM motion, this question is going to land on your desk in the next quarter whether you ask for it or not. Here is the map.
What Is the AI Agent Rediscovery Problem?
The pattern. Agents re-do work they already did.
An agent runs a task. It pulls the customer record. It checks the policy. It drafts the response. The next run, it does all of that again from scratch. It re-fetches the same record. It re-reads the same doc. It re-asks the user for something the system already has.
Token budget burns before useful work starts. The agent feels sluggish and forgetful even though the underlying model is fine. The wrong primitive at the foundation forces every feature on top to compensate.
Watch for agents that look great in a one-shot demo and fall apart on the third consecutive run. That is the rediscovery tax compounding.
Why Is Vector Search Insufficient for AI Agents in Production?
The chatbot era ended. The retrieval contract did not change with it.
Vector search was built for one job. Question answering. A user asks how do I reset my password. The system embeds the question, pulls three semantically similar chunks from the help center, the model writes a paragraph. That loop works because the answer lives in a couple of paragraphs and the user does not need exact wording.
Agents do not work like that. An agent does not ask a question and stop. It runs a task. It needs the customer record plus the policy plus the entitlement plus the prior tickets all assembled into the right shape. It needs a bundle of operating context, not three relevant paragraphs.
Pine Cone’s own framing on Nexus is that agents need retrieval that carries intent, filters, access policy, provenance, response shape, confidence, and budget. Vectors carry similarity. That is one signal of many.
Watch for teams that pick a vector database and then spend six months wiring everything else around it. The database is determinative. If you pick it before you know what your agent needs, you are constraining the agent to whatever the database is good at.
What Are the Four Shapes of Memory an AI Agent Needs?
Most agents need more than one shape. No single primitive delivers all of them.
Credit to Nate B Jones, whose breakdown of the memory wars pulled this thread for us. Here are the four shapes the industry is racing to support.
1. Fuzzy prose. Help center docs. Internal wikis. Email threads. Vector search is the right primitive here. This is the original RAG.
2. Long structured documents. Contracts, financial filings, policy docs. The structure carries meaning. Chunking flattens the meaning out. Page Index, one of the newer approaches, builds a hierarchical tree of the document and reasons through it instead of embedding it. They claim 98.7 percent accuracy on FinanceBench. The principle holds even if the specific vendor does not. Tree of document beats chunked vector when structure matters.
3. Business data in tables. Customer records, opportunities, usage events, governed metrics. This is most of your CRM and warehouse. You cannot reliably reason over churn risk or supplier risk or renewal forecasting by turning a spreadsheet into prose and asking a language model to read it. You need a semantic layer and ideally a tabular reasoning model. SAP’s billion-euro bet on Dremio and Prior Labs is exactly this.
4. Relationships. Which suppliers connect to which shipments. Which incidents trace back to the same root cause. Which accounts share a particular failure pattern. These are graph questions. Chunks and tables cannot carry them. Microsoft GraphRAG is the most prominent attempt to handle this shape today.
If your agent’s job involves more than one of these shapes (most GTM agents do), the answer is not to pick the best vector database. The answer is to pick the right primitive for each shape and assemble them.
Watch for teams that flatten everything into one shape because they only have one tool. The model has to compensate with expensive tokens for every mismatch.
How Do You Architect Memory for a Production AI Agent?
Stop picking the database first. Write the bundle.
A bundle is the specific set of fields your agent needs to do its job. Not relevant context. The actual fields.
For a customer expansion agent, the bundle might look like this.
- Customer record. Table.
- Current plan and entitlement. Table.
- Usage data for the trailing 90 days. Table or time series.
- Prior support tickets. Prose, vector search.
- Last three call transcripts and meeting notes. Prose, vector search.
- Account hierarchy and parent-org relationships. Graph.
- AE comp structure for whoever owns the account. Table.
- Permission scope for whether the agent can act or only recommend. Governance.
Notice how no single primitive delivers all of that. Vector search alone gives you transcripts and tickets. A graph alone gives you account hierarchy. A semantic layer alone gives you the governed business data. The agent’s actual job is assembling and reasoning over the bundle. Search is the small part.
When you write the bundle first, three things happen. You see that most fields do not live in one system. You see that some of them need to be governed, not just retrieved. You see that what your agent really does is assemble and reason.
Then you pick primitives that deliver the bundle. Sometimes that includes a vector DB. Sometimes a knowledge graph. Sometimes a semantic layer over the warehouse. Usually a mix.
Watch for the trap of treating “more context window” as the architecture answer. Chroma published research showing model performance degrades as context grows larger and more cluttered. The goal is appropriate context, not maximum context.
Where Do You Start Without Buying Anything New?
The data is in your existing agent logs.
Look at your agent runs from the last 30 days. Count the retrieval calls before useful work starts. Note how often the agent opens the same sources. Track how much of the token budget is sucking in raw context. Count how often the agent asks the user for something the system already has. Count how often the next run rediscovers what the prior run learned.
The pattern is already in your runs. Once you see the bundle your agent keeps trying to assemble on the fly, you have your blueprint.
Watch for the urge to start a vendor RFP before you have read your own logs. The cheapest insight in this whole space is free.
The Pattern Across the Memory Layer
Every infrastructure vendor moving on memory right now is solving for a different shape. Pine Cone is rewriting retrieval to carry more than similarity. Page Index is keeping document structure intact. SAP is buying the tabular and governed-data shape. Microsoft is pushing graphs. They are not competing for the same slot. They are each solving one shape of memory that production agents need.
The teams that win the next 18 months are not the ones that shop the most fashionable retrieval product. They are the ones who wrote down what their agent needed to do the work before they wrote a check for any database.
Build the bundle first. The primitives follow.
FAQ
What percentage of AI agent compute is wasted on rediscovery?
According to Pine Cone, up to 85% of agent compute can be spent on rediscovery. That includes re-fetching records the agent already retrieved, re-reading documents the agent already summarized, and re-asking the user for information the system already has. The percentage published in the Pine Cone Nexus launch material is the upper bound, but operators running production agents widely report that the share is high.
Why is Pine Cone saying vector search alone is not enough for AI agents?
Vector search was designed for chatbot-era question answering. One question, three semantically similar chunks, one paragraph response. Production agents do not ask one question. They run multi-step tasks that need a bundle of operating context, including intent, filters, access policy, provenance, response shape, confidence, and budget. Pine Cone’s new product, Nexus, with its query language NoQL, is an attempt to make retrieval carry that broader contract rather than only similarity.
What are the four shapes of memory an AI agent needs?
Production AI agents typically need to retrieve four shapes of memory. Fuzzy prose (help center docs, wikis, emails) which fits vector search. Long structured documents (contracts, financial filings) which fit hierarchical tree approaches like Page Index. Business data in tables (CRM, warehouse, governed metrics) which fits semantic layers and tabular foundation models like SAP’s Prior Labs. Relationships (account hierarchies, incident lineage, supplier networks) which fit graph approaches like Microsoft GraphRAG.
What is the bundle pattern for AI agent memory?
The bundle pattern is the practice of writing down the specific fields your agent needs to do its job before picking any retrieval database. Each field gets a source, a freshness requirement, a permission scope, and a primitive that delivers it. Most production agents need a mix of vector search, semantic layer, and graph queries because their work spans multiple shapes of memory. The bundle pattern surfaces that mix early instead of letting the agent rediscover it at runtime.
Does a bigger context window solve the AI agent memory problem?
No. Chroma has published research on what they call context rot. Model performance degrades as the context window grows larger and more cluttered. A bigger window gives the model more room to work, but it does not decide what belongs in that room, mark which source is authoritative, enforce permissions, preserve document hierarchy, or distinguish memory the user confirmed from memory the model inferred. The goal for production agents is appropriate context, not maximum context.