AI Software Development
LLM consulting and application engineering
Model selection, prompt architecture, eval harnesses, observability, and token-cost control
- Duration
- 3 to 6 weeks
- Ladder stage
- Product
- NATIVE stages
- Integrate, Validate
The situation
When this engagement is the right one
Model choice is a cost and quality decision, made against your own evaluation set, revisited on a cadence. Made once from a benchmark table, it is a guess that gets expensive at volume.
You are probably seeing
- Prompts live in application code and nobody can tell what changed when quality dropped.
- Token spend is growing faster than usage and nobody can attribute it.
- Model selection was decided by whoever had an account.
What we do
The work, in the order it happens
- 01Build a task-specific evaluation set and score candidate models against it.
- 02Restructure prompts into versioned, testable artefacts.
- 03Instrument traces, latency, cost per task, and quality signals.
- 04Set caching, routing, and fallback policy.
- 05Establish the review cadence for revisiting model choice.
What you get, and keep
- A model selection report against your own evaluation set, with cost per task.
- A versioned prompt architecture in your repository.
- Observability and cost attribution instrumentation.
- A routing, caching, and fallback policy.
Prerequisites
- A working application or prototype to instrument.
Not included
- We do not train foundation models.
- We do not recommend a provider without an evaluation on your own tasks.
Where this sits in the method
Commercially this is a Product engagement on the Proof, Product, Platform ladder.
FAQ
Questions we get asked
- Which model should we use?
- The one that clears your quality bar at the lowest fully loaded cost per task, measured on your evaluation set. That answer changes; the method does not.
- Do you recommend open models?
- Where data residency or cost curve justifies the operational burden. We will state the burden honestly.
Related engagements
Work that usually sits either side of this
AI Software Development
Enterprise RAG and Knowledge Systems
Retrieval architectures designed to survive audit: source attribution, permission inheritance, and evaluation
AI Governance and Risk
Model Risk and Assurance
Evaluation harnesses, quality bars, drift monitoring, and audit-ready documentation for regulated workflows
Next step
Ready to scope llm application engineering?
Bring the pilots you already have running. The first call is a scoping conversation, not a pitch.