Language models in production, with a score to beat.
Engineers who build on Claude and OpenAI models with typed interfaces, evaluation sets, cost budgets, fallbacks and monitoring, so the demo becomes a system you can operate.
- Claude
- OpenAI
- Azure OpenAI
- AWS Bedrock
- Python
- FastAPI
Large language models changed which problems are worth solving in software.
Extraction from messy documents, classification of free text, summarisation of long records, conversational access to systems that used to need a trained operator — all of these were expensive projects five years ago and are now a well-engineered API call. We build on Anthropic’s Claude models and OpenAI’s GPT models because they are the providers we run primarily, and both offer mature tool use, structured outputs and enterprise data-handling terms.
What has not changed is what makes software dependable. A model is a probabilistic component with a latency, a cost and a failure mode, and we treat it as one: a typed interface in front of it, an evaluation set behind it, a timeout, a fallback and a dashboard. The same evaluation discipline gates the models inside our own products — the next-best-offer and tier-prediction models in Loyalty & Retention do not ship without a score to beat.
We do not put a language model where a rule, a lookup or a classical model already gives the right answer more cheaply and more predictably. Use-case triage comes before the first prompt, and “no model here” is a result we deliver in writing.
Six things we ship on Claude and OpenAI.
Document extraction
Invoices, contracts, claims, shipping papers — turned into validated, typed records with structured outputs, a confidence signal per field and a human review queue for the ones the model is unsure about.
Retrieval-augmented answering
Grounded question answering over policies, manuals and case histories, with retrieval on pgvector or Pinecone, citations that resolve to a real passage and a measured answer-quality score before launch.
Copilots inside existing products
Assistants that read the screen the user is on, call the same APIs the UI calls, and propose actions the user confirms — tool use with permissions, not a chat window bolted onto the side.
Classification and routing
Support tickets, emails, complaints and transactions labelled and routed at scale, with a labelled evaluation set that tells you the precision before the model touches a live queue.
Agentic workflows
Multi-step processes — reconcile, look up, draft, escalate — where the model plans and calls tools under a budget, with every step logged and every side effect behind an idempotent API.
Summarisation and drafting
Long-record summaries for agents and case workers, first drafts of correspondence, release notes from commits — with the source material attached so a reader can check the model’s work.
When to choose them.
Choose a language model when the input is unstructured — text, documents, conversation — and the task is one a competent person could do with the same material in front of them. Choose Claude or OpenAI specifically when you need mature tool use, structured outputs, long context and an enterprise data-handling posture from the provider. Choose an evaluated build when the feature will face customers or drive decisions, because a score on your own data is the only honest basis for launching.
Do not choose a language model for arithmetic, deterministic rules or lookups that a database answers exactly; for latency-critical paths where a few hundred milliseconds is too slow; or where the output cannot be checked and a wrong answer is expensive. And do not choose a provider from a leaderboard — choose it from your evaluation set, at your cost budget, with the application written so the provider is a configuration you can change.
A structured extraction call with a schema, a timeout and a fallback.
class Invoice(BaseModel):
supplier: str
total: Decimal
currency: Literal['TRY', 'EUR', 'USD']
confidence: float = Field(ge=0, le=1)
async def extract(doc: str) -> Invoice:
# provider and model come from config — the eval suite is what lets us change them
async with asyncio.timeout(settings.llm_timeout_s):
raw = await llm.complete(prompt=EXTRACT_V7, input=doc, schema=Invoice)
inv = Invoice.model_validate(raw)
if inv.confidence < settings.review_threshold:
await review_queue.enqueue(doc, inv)
return invWhere the model sits in an Odeontech estate.
What the LLM features we hand over look like.
- An evaluation set before a prompt. Real, labelled examples from your data — frozen, versioned and run in CI. A prompt or model change that lowers the score does not merge.
- Typed at the boundary. Structured outputs validated against a schema; a malformed response is a handled error with a retry, not a string parsed with a regular expression.
- Provider as configuration. One thin client interface, prompts versioned in the repository, and provider, model and parameters set per environment — so switching is a pull request with an eval result attached.
- A cost model before launch. Tokens per request measured on the eval set, caching where the context repeats, a budget with an alert wired to it. You know what a thousand requests cost before the first user does.
- Timeouts, fallbacks, degradation. Every call is bounded; a slow or failing provider degrades to a cheaper model, a cached answer or a human path — the feature is built not to take the page down with it.
- Guardrails that are tested. Input screening, output validation, refusal paths and escalation are behaviours in the eval set, with the adversarial cases included.
- Traced end to end. Prompt version, model, tokens, latency and the retrieval hits on every request in OpenTelemetry, so “why did it say that” has an answer.
- Data handled deliberately. Enterprise endpoints with zero retention, PII redaction before the call where policy requires, regional routing for KVKK and GDPR, and a written record of what leaves the estate.
What usually ships alongside it.
- A versioned evaluation suite with a stored baseline, wired into CI as a merge gate
- A thin, typed LLM client with provider routing, retries, timeouts and structured-output validation
- A retrieval pipeline with chunking tuned on your documents and retrieval quality measured separately from answer quality
- A cost and quality dashboard per feature: tokens, latency, score, refusal rate, human-review rate
- A model and provider decision record with benchmarks on your data, and runbooks for outage, drift and quality incidents
Frequently asked questions
Neighbouring technologies and the service that uses them.
Need engineers who ship LLM features to production?
Tell us the use case, the data constraints and the seniority you need. We will propose a named team or place specialists into yours — and tell you honestly if a model is not the right tool.