Retrieval and data
A model that does not know your content will confidently make it up. Retrieval fixes that by putting the right passage in front of it, and the whole job is being able to prove the right passage was found.
01Why
Most retrieval projects fail on measurement, not on embeddings
Retrieval quality is measurable. We measure it, before we build, after every change, and on a schedule once it is live. That is the difference between the projects that hold up and the ones that quietly stop working.
Before anything is built we agree a question set drawn from real questions, and we score the current system against it. Every change after that has a number to beat. When someone asks whether the new chunking helped, the answer is a score, not an opinion.
Your documentation gets rewritten, products get renamed, a policy page moves. A retrieval system nobody measures degrades quietly, because a wrong answer looks exactly like a right one. The scores and the staleness alarm are what make that visible.
Which vector store you use matters far less than how your content is split, how the shortlist is ranked, and whether exact terms survive the search. We pick a store that fits your stack and spend the time on the parts that move the score.
Most of this work sits underneath an agent, as the tool it calls when it needs to know something. Grounded answers with citations are what let an agent act on your content instead of paraphrasing the internet.
02Wire
What the pipeline is made of
Seven parts, each of which can be measured on its own. We build them in this order and re-score after each one.
- ingestion
- Connectors to the sources you actually use, with change detection so re-indexing is incremental rather than a nightly rebuild of everything.
- chunking
- We split on document structure rather than a fixed character count, so a retrieved passage is a whole thought instead of half a sentence and half of the next one.
- embeddings
- Chosen for your content and your language, and re-run when the model changes. That is a migration we plan for rather than discover.
- hybrid retrieval
- Keyword and vector search together, because exact terms like a SKU or an error code fail on vectors alone.
- reranking
- A second pass over the shortlist, which is usually the cheapest large accuracy gain available in the whole pipeline.
- citations
- Every answer carries the passages it came from, so a person can check it rather than trust it.
- access control
- Retrieval respects who is allowed to see what. A search index is not a license to leak, so permissions are applied at query time, not hoped for at ingest.
03Build
How we build it
Measurement comes first and stays first. Nothing gets added to the pipeline without a score on either side of it.
-
Inventory the sources
We list every source that could answer a question, then agree in writing what is in scope and what is deliberately left out. Out of scope is a decision, not an oversight.
-
Build a question set from real questions
We collect questions people already ask, in the words they ask them in. That set is the test set for the rest of the project.
-
Measure the baseline before changing anything
We score whatever you have today against the question set. Without that number, every later improvement is a claim rather than a result.
-
Build ingestion, chunking and hybrid retrieval
Each part lands on its own and we re-measure after it, so we know which change moved the score and which one did nothing.
-
Add reranking and citations
A second pass narrows the shortlist, and every answer starts carrying the passages behind it so a reader can check the source.
-
Set the refresh schedule and the staleness alarm
We agree how often each source re-indexes and what fires when a source stops arriving, so silence is treated as a fault rather than as good news.
What this usually gets built alongside
Retrieval is rarely the whole job. It is the tool an agent calls, or the grounding under a workflow that drafts something a person signs off.
04Trust
How retrieval is measured
These are the numbers we report on your content, with your question set. We will not quote you a score from someone else's corpus, because it would not tell you anything about yours.
- recall@k
- Did the right passage make the shortlist at all. If it never gets retrieved, no amount of prompting will recover it.
- precision@k
- How much of the shortlist was noise. Noise costs tokens and gives the model room to pick the wrong passage.
- groundedness
- Is every claim in the answer supported by a retrieved passage, rather than by the model's own recollection.
- answerable and unanswerable
- The set includes questions your content cannot answer, and the system is scored on saying so rather than inventing.
- freshness
- How long after a source changes the index reflects it, measured per source rather than averaged into one comforting number.
05Asked
Questions we get asked first
Do we need a vector database?
Not always. Hybrid retrieval needs a keyword index and a vector index, and plenty of stacks already run one or both. We pick the store that fits your content, your hosting and your team, and we say so plainly when what you already run is enough. The database is the least interesting decision in the project.
What happens when our content changes?
Ingestion detects what changed and re-indexes only that, so an edit to one page does not trigger a full rebuild. We agree a refresh schedule per source and set an alarm on staleness, so a connector that quietly stops feeding is noticed rather than trusted.
Can it say it does not know?
Yes, and we test for it. The question set includes questions your content cannot answer, and the system is scored on saying so rather than inventing. Every answer it does give carries the passages it came from, so a person can check the source instead of taking the model's word for it.
Bring us the questions people keep asking
Send a handful of real questions and tell us where the answers live today. We will tell you what a retrieval system would have to reach, and how we would score it.