Training agents to do professional work requires a different kind of data than training models to answer questions. This note describes how we think about that difference and how it shapes the way we work with practitioners.
Outcomes are not trajectories
Most datasets map a prompt to an answer. That shape was adequate when the goal was a single correct reply. It is inadequate when the goal is an agent that can do a task the way a careful professional does it, because professional quality lives in the path, not only in the destination.
A tax opinion, a diagnostic workup or a contract review is valuable because of the sequence in which information was gathered, the alternatives that were weighed and quietly set aside, and the moment at which the practitioner committed to a reading. Two professionals can reach the same conclusion with very different levels of rigour, and an answer only dataset cannot tell them apart.
We therefore treat the unit of work as a trajectory: an ordered record of steps, actions, evidence consulted, decision points and a final artifact. The answer is one field in that record, not the record itself.
Calibrated difficulty
Task design is where most expert data efforts succeed or fail. Our principle is calibrated difficulty. A task must be hard enough that a frontier model fails on it with meaningful frequency, otherwise the expert adds nothing the model did not already know. It must also be verifiable enough that correctness is decidable, otherwise review becomes a matter of taste and agreement between reviewers collapses.
In practice this means we write tasks against current frontier models, not against an abstract idea of difficulty. Before a task family enters production, we probe it ourselves. Tasks that every model solves are discarded as too easy. Tasks where competent reviewers cannot agree on what correct means are rewritten until the rubric converges, or discarded as unverifiable. The productive band sits between those two failure modes.
Externalising decisions
Experts rarely narrate their own reasoning unprompted. Years of practice compress judgment into what feels like intuition, and asking someone to simply explain their thinking produces retrospective rationalisation rather than the reasoning itself.
Our instruments instead ask practitioners to externalise decisions as they work. At each decision point the practitioner records what they decided, which alternatives they seriously considered and rejected, why those alternatives lost, and the exact point at which they committed. The rejected options matter as much as the chosen one: they teach a model where the boundaries of the decision are, not just where the centre is.
This is slower than answering questions, and deliberately so. The value of the data is concentrated precisely in the places where the practitioner had to slow down.
Synthetic but realistic cases
We do not use real client, patient or case files, ever. Every task is built on a synthetic case: constructed to carry the texture, ambiguity and stakes of real work, while containing no third party personal data and no confidential material.
Synthetic construction has a second advantage beyond compliance. Because we author the case, we know the ground truth about what the case contains, which makes correctness decidable and review auditable. A reviewer can check a step against the case record rather than against their own experience of a similar real matter.
Realism is enforced through review by practitioners in the same specialty, who flag anything that reads as artificial, dated or institutionally implausible before the case reaches contributors.
What we deliver
Each trajectory ships against a schema with five parts:
- steps: the ordered narrative of the work as it unfolded
- actions: the concrete operations performed at each step, such as a search, a calculation or a comparison
- evidence consulted: the sources, documents or case elements the practitioner examined, in the order they examined them
- decision points: each commitment, the alternatives that were rejected, and the stated reason for rejection
- final artifact: the deliverable a client would receive, such as an opinion, a plan or a reviewed document
The schema is versioned and stable within a batch, so downstream teams can build training and evaluation pipelines against it without surprises.
Where this leads
We believe the next gains in professional grade agents will come from data that teaches judgment, not just answers. If your lab is working on agentic capability in a regulated or high stakes domain, we would be glad to discuss a scoped pilot. Book a data consultation and we will walk through your target capability together.