How the system is assembled.

We deploy open-weight models through vLLM, Ollama or llama.cpp, depending on the hardware and the throughput you need. vLLM suits a dedicated GPU host with continuous load. Ollama is the right default for a single-node deployment that must be operable by your own staff. llama.cpp covers CPU-only or mixed environments where a GPU is present but not always available.

Sizing starts from your document types and concurrency, not from a model leaderboard. A 7B–14B model at Q4 or Q5 quantisation will serve most narrow document work — contract abstraction, variance narrative, code search — at acceptable latency on a single consumer or datacentre GPU. We size up when the held-out evaluation shows the smaller model missing cases your reviewers care about, not because a larger number appeared on a benchmark.

Quantisation is a trade-off we document explicitly. Q4_K_M is usually the starting point: half the VRAM of full precision with a small recall cost on rare tokens. Q8 is reserved for tasks where a single misread clause or figure is unacceptable and the hardware can carry it. The point is not to run the largest model you can afford. It is to run the smallest model that passes your evaluation on your material.

For narrow document work, a well-fitted small model beats a frontier API. The API model is general; your corpus is not. Retrieval plus a house adapter on a 14B base will outperform a 70B general model on questions whose answers live in your own index, because the retrieval step does the heavy lifting and the model's job is synthesis in your register, not world knowledge.

Your documents stay where they are. We mount the vault read-only and build an index beside it — embeddings, structured metadata, and a keyword layer — without copying the corpus to a third party or a shared tenancy.

Chunking follows document structure rather than a fixed token count. A contract is split at clause boundaries. A ledger export is split at account and period. Source code is split at function and module, with imports and call sites preserved in the metadata so cross-file questions still resolve. Fixed-size chunks produce citations that point at the middle of a paragraph; structure-aware chunking produces citations a reviewer can open and verify.

Search is hybrid: dense vector retrieval for paraphrase and concept, BM25 for identifiers, dates, amounts and proper nouns. Results are fused and re-ranked before they reach the model. The model never sees the whole corpus; it sees the passages retrieval selected, which is why hallucination on sourced questions drops sharply compared with a raw completion.

We do not ask the model to cite. Citations are assembled from the retrieval trace: each passage carries its source file, page or locator, and the answer template binds claims to those locators. If retrieval returns nothing above threshold, the system says so rather than inventing a source. That behaviour is enforced in the pipeline, not requested in the system prompt.

General models write in a generic register. Your firm does not. We train low-rank adapters — LoRA or QLoRA, depending on VRAM — on material you designate: redacted matters, anonymised packs, approved correspondence. Training runs on the same hardware as inference. No weights and no training data leave the boundary.

Evaluation is against a held-out set the firm defines before training starts. Typical cases: ten questions whose answers are known and documented, five document types the model must handle, three failure modes you have already seen from hosted tools. We report precision on sourced claims, refusal rate when retrieval is empty, and register drift measured against your style guide. A model that passes on our generic set but fails on yours does not ship.

Adapters are versioned alongside the base model. Rollback is a configuration change, not a retraining exercise. When your corpus shifts — a new reporting format, a merger, a regulatory change — we retrain on the delta and re-run the held-out set before the new adapter is promoted.