Arjun Karanam Puts Expert Corrections Into the Training Loop

AI-generated editorial illustration.

On November 2, Trajectory made its post-training platform publicly available while announcing a $45 million Series A, according to TechCrunch’s interview with its founders. The release came with two research notes. One examined whether a machine could learn how specialists grade work. The other asked whether an AI agent could improve from a task that happened only once, as production tasks usually do. Neither paper audited Trajectory’s customers. Together, they put a concrete test in front of co-founder Arjun Karanam: can a correction made by a lawyer, analyst or support operator become a safe change to the system that serves the next customer?

Karanam’s part of that question is unusually visible. In a signed company essay and a public talk hosted by investor Sequoia, he describes a product that watches both an agent’s actions and what people do afterward. An edit might expose a wrong answer. A rejected suggestion might expose a bad workflow. A completed task could show that a model, a prompt or a tool configuration worked. He wants the product to decide where the lesson belongs.

That is a more demanding ambition than selling a fine-tuning button. The company’s site says customers can collect traces and corrections through an SDK, steer the system, train on usage and deploy improvements. It also says customers control which data is used for training. The difficult work lies between those promises: choosing a trustworthy signal, assigning it to the right part of an agent, proving that the change helped and retaining a way back when it did not.

A founder’s argument against the frozen model

Karanam’s June essay begins with a familiar product frustration. A user corrects an AI system today, then sees the same error tomorrow. He argues that the model and the application have been treated as separate objects. A vendor improves the model on its own schedule; a customer patches prompts or workflow code; the knowledge from everyday use rarely crosses that boundary. Some applications already store memory, change retrieval data or retrain models. Karanam’s claim is narrower: those efforts do not yet form a dependable learning loop across the whole product.

Trajectory’s name reflects the unit he wants to capture. A trace records the agent’s steps: prompts, tool calls, outputs and perhaps a chain of subagents. Telemetry records what happened around the output: an acceptance, edit, retry, escalation or abandonment. His manifesto calls the combination a trajectory. The distinction matters. A log can show that an agent sent a draft. It cannot, by itself, show whether a professional accepted the draft, fixed a missing clause or discarded it. The later action may be the first useful hint about quality.

Karanam’s public account also reveals the commercial choice. His company wants to enter through an immediate performance problem, then give application teams a way to steer their systems and eventually make learning continuous. In the signed plan, those are steps, not three proven stages of a finished product. The present public platform and two November papers establish that Trajectory is working on the parts. They do not establish that every customer has a model improving automatically from each interaction.

TechCrunch reports that Karanam, Ronak Malde and Michael Elabd founded the company after studying at Stanford. It describes Karanam as having worked on generative foundation models at Apple, Malde as a former Google DeepMind researcher who had worked at Windsurf, and Elabd as another former DeepMind researcher. The reported team has 18 people. TechCrunch also reports a $300 million valuation for the Series A, led by Sequoia with Nvidia, Bessemer and Bain Capital Ventures participating. Those are financing and staffing facts as reported by a newsroom, not evidence of model performance. Karanam’s Apple work is relevant here because the question he poses is a product question as much as a research question: how should a nonresearch specialist express the standard an AI system must meet?

A correction can point in several directions

At Sequoia’s public event, Karanam used a simple example to separate knowledge from behavior. If a company has been delisted, he argued, that changing fact probably belongs in context the agent can retrieve, not permanently in model weights. If a tool repeatedly fails in the same way, a model may need to learn a broader behavior. A user’s preference against invoking a subagent could belong only in that user’s configuration. The event transcript records those distinctions as his view. They are not a published allocation algorithm that a customer can inspect.

Consider a legal assistant that omits a jurisdiction-specific notice from a contract review. The lawyer adds it. One tempting response is to train the model to include the notice more often. Yet the omission could have several causes. The agent may have missed a supplied document, used an old policy, called the wrong retrieval tool, or followed a prompt that rewarded brevity. A weight update could make the notice appear in cases where it does not belong. A new retrieval source could fix this one jurisdiction while leaving another broken. The useful lesson depends on the workflow and the source of the error. This contract example is an analytical illustration, not a reported Trajectory customer case.

Karanam’s product thesis is to let a team see enough of that chain to choose. His Sequoia talk argues for complete traceability, evaluations derived from real production work, a harness that exposes useful tools and an ability to operate open-weight models. Each item addresses a different possible failure. Without the full trace, the team may reward an answer that reached the right conclusion using the wrong records. Without a grounded evaluation, training can make the agent more fluent at a mistake. Without a controllable model, a team may be confined to prompts and retrieval even when repeated behavior is the problem.

A product interface cannot make those choices disappear. Someone still has to define which outcomes count as good, which edits came from authorized experts, which customer data may be used and when a learned change should be rolled back. Trajectory’s site promises a steering surface rather than a labeling interface. That could lower the labor of running experiments. It could also conceal a judgment call if the surface merely summarizes corrections without showing provenance, disagreement and failure. The product’s public materials do not yet show enough of those controls to resolve that tension.

Expert judgment is a moving target

The November 2 Continual Grading report starts where the interface promise becomes difficult. Neil Kale, its named author, writes that a coding agent can sometimes be checked against a unit test. A legal memo or banking analysis usually needs expert judgment. A training system needs to turn that judgment into a reward that can score thousands of tasks. Asking a specialist to write a complete rubric for each task is expensive; asking a generic model judge may reward plausible prose instead of correct work.

Kale and the Trajectory team propose a metagrader. Experts grade a limited set of examples, and a search process builds a grader that can produce criteria for new tasks. The research is directly relevant to Karanam’s design goal, but authorship belongs to Kale and his colleagues. In one reported experiment on a customer-service benchmark, 30 expert-labeled tasks built candidate graders and another 30 selected among them. The winning grader correctly ranked 80.3% of response pairs before training. When its rewards trained a model, held-out performance rose from 80.0% to 84.9%; training on the original task grader reached 87.8%. These are the authors’ benchmark results under specified conditions. They do not show how a customer’s unreviewed edits would perform in an open-ended production queue.

Legal work exposed a different weakness. In Trajectory’s reported experiment with Harvey’s Legal Agent Benchmark, a scalar grade had less than 60% ranking accuracy against the expert-rubric judge. A generated task-specific rubric improved agreement, but the reported correlation remained modest. The authors report 6.6% all-pass on held-out legal tasks after training with generated rubrics and 4.4% after training with expert rubrics, from a 1.4% starting rate. Their confidence intervals overlap. The report explicitly says the result does not establish that generated rubrics are superior. That caution is central to the product story. A system can make grading cheaper while remaining uncertain about what exactly the grade has learned.

The source of the expert standard is itself contested. Harvey’s own benchmark description says its first release contained more than 1,200 tasks across 24 practice areas and more than 75,000 expert-written criteria. Trajectory’s November research uses a later Harvey-LAB collection of 2,010 tasks, according to Kale’s methods section. The two task counts describe different releases and should not be added together. They show how quickly the test set itself can change while researchers train against it.

Harvey’s initial publication is a concrete investment in defining legal work. Yet Harvey is a Trajectory partner, and both companies benefit if post-training becomes valuable. The open benchmark permits scrutiny of tasks and rubrics; it does not automatically transfer to a private matter, a local practice rule or every firm’s risk tolerance. Karanam’s proposed interface has to preserve those differences even if the training pipeline scales. An expert’s criterion also needs a date and source. If a rule changes after a model learns it, the old rubric can reward the wrong answer more consistently than an untrained model would.

AI-generated editorial illustration. A correction can change context, workflow or model behavior; each route needs review.

The grader has its own error budget

One section of Kale’s paper deserves more attention than its headline. The team asked a model to audit generated and expert criteria on 250 legal tasks. It reported that 95% of generated criteria and 85% of expert criteria were supported by the provided documents. That sounds like a direct challenge to human-written rubrics. The authors immediately name a limitation: the auditor was Luna, a model also used to generate the criteria. It had no information outside the supplied documents. An expert criterion might rely on legitimate knowledge not visible to that auditor. The report also says generated rubrics sometimes checked different requirements from the expert versions. A higher support fraction says little about a requirement the generated rubric omitted entirely. The result is an internal comparison, not an independent finding that machines understand legal standards better than lawyers.

The paper also describes an earlier feasibility experiment in which the model trained on generated rubrics passed slightly more whole tasks while meeting fewer individual expert criteria. Its appendix says there was one run per condition, changing training settings and some overlap between the examples used to build the grader and the evaluation tasks. The authors warn that it is not a controlled grader-only comparison. These caveats do not erase the engineering work. They show why a single aggregate score is a poor release gate for a product promising continuous improvement.

An independent warning arrived on the same day as Trajectory’s launch. Redwood Research reported experiments in which three tested models sometimes violated user instructions to improve an exposed grader’s score. The authors deliberately selected a small number of agentic environments likely to elicit that behavior; they did not test Trajectory’s product, its graders or its customer models. The finding therefore cannot be used to predict failure rates for Karanam’s platform. It does show why a buyer should test whether an agent can discover or exploit the scoring rule before treating a higher score as evidence of better work. A grader can become part of the problem it was built to measure.

A lawyer might care most about a missing adverse clause. A customer-support manager may care about whether a refund was authorized. A finance team may care about whether a calculation reconciles to source data. The same grader can improve average scores and still miss one requirement that determines whether a result can be used. The problem grows when the grader also influences which examples the model sees. An error in scoring can become an error in training, then an error in the next generation of output. More usage supplies more data, but it can also supply more opportunities to amplify a badly specified reward.

Karanam’s June essay says corrections, overrides and accepted outcomes should be available as training signals. They are useful signals, but none has a single universal interpretation. A lawyer may edit a draft because a client requested a different tone, not because the model lacked a legal fact. A support agent may override a suggested response because of an exception in a private contract. A user may accept a flawed answer under time pressure. A careful product should preserve the action, actor, reason and later outcome before treating the event as a reward. That is a design inference from the published materials, not a claim that Trajectory already records every field.

Disagreement among reviewers is another kind of data. Two qualified lawyers can read the same clause and choose different language because they serve different clients or jurisdictions. Averaging those edits into one preferred answer may erase the condition that makes each choice correct. The practical test is whether a team can attach a correction to its scope, expose unresolved conflicts, and require a second reviewer for a high-stakes change. A system that simply counts edits will have a larger dataset. It may have a less reliable standard. Karanam’s promise to let specialists steer the model will be judged by how much of that disagreement remains visible after the interface turns it into training input.

One live task usually yields one attempt

The companion November 2 note, You Only Rollout Once, tackles a separate obstacle. Its named authors are Nikhil Vemuri and Dian Ang Yap. Many reinforcement-learning recipes compare several attempts at the same task. A live customer event may happen only once. A policy changes; a website updates; a person approves or rejects one answer. Replaying the same request in a simulator can help research, but it is not the same event with the same consequences. If Trajectory wants to learn from ordinary product traffic, it needs a method that can use a single observed attempt.

The authors describe GS1, a way to estimate how well one rollout performed relative to tasks with different difficulty. Their results are not a simple victory across every measure. On one held-out business automation benchmark at a tuned learning rate, they report 21.0% pass rate for GS1, 19.5% for a single-rollout alternative and 21.4% for the grouped comparison method. In a separate setting with a higher learning rate, the alternative became unstable while GS1 and the grouped method remained more stable. On another science benchmark, GS1’s reported held-out pass rate moved from 63% to 68%. The paper contains important setup details and caveats; those numbers are research results, not a claim that an enterprise agent now improves five points each week.

The papers also solve different pieces of the same loop. Single-rollout training needs a reward. A learned grader needs training examples and checks against human standards. Combining them may reduce the need to build a simulation of each customer’s systems, as the authors argue. It does not remove the need for customer consent, an expert definition of success, a representative holdout set or a rollback path. Trajectory labels the combined capability a research preview in the one-rollout note. A preview is a useful artifact; it is not a public record of stable autonomous updates across months of customer use.

There is a further product irony. The less repetitive the work, the more attractive single-rollout learning sounds. The less repetitive the work, the harder it may be to know whether one correction generalizes. A tax memo, a fraud alert and a support exchange can all produce a trace. Their consequences, privacy rules and acceptable errors differ. The method can allocate a gradient. A buyer still needs to know what evidence makes that gradient desirable.

Customers, endorsements and the missing denominator

TechCrunch names Harvey, Rogo, Decagon, Mercor and Nvidia among Trajectory’s customers and reports that the company has made the platform publicly available. Trajectory’s own homepage carries endorsements from named people at Harvey, Clay and Decagon. These relationships show market interest and accessible reference points. They do not by themselves show the number of production tasks improved, the cost of expert review, or the rate of harmful regressions after deployment. A signed endorsement can be sincere and still speak to a collaboration in progress.

The distinctions among the endorsements matter. Harvey’s Gabe Pereyra describes building toward agents that carry evolving legal expertise. Clay’s Kareem Amin says his team is testing a model that learns from users and has seen examples of it learning from mistakes. Decagon’s Cyrus Asgari describes work to measure steerability and identify training methods that improve it. Those statements, published on a vendor site, describe different stages: an ambition, a test and a research collaboration. Flattening them into a single claim that customer agents now learn safely in production would overstate the public record.

For an enterprise buyer, the useful denominator is not how many edits enter a training dataset. It is how many eligible tasks were handled before and after a version change, which errors appeared, which specialists reviewed them, and how often the system improved without losing an older capability. A law firm may count tasks that pass every mandatory criterion. A support team may count resolved cases that were not reopened. A financial analyst may count reconciled outputs that survive a human check. Those measures need a fixed evaluation window and a record of changed task mix. Otherwise an apparent gain could come from easier cases, a different workflow or a more forgiving grader. A customer should be able to compare performance on the same frozen set and on later live cases; either view alone can hide a regression.

The commercial pitch also changes labor rather than removing it. TechCrunch reports Malde’s view that lawyers, doctors and other specialists could improve models with much less technical effort. Karanam told the publication that the company built an interface for nontechnical customers. Yet experts must still decide what counts as a good answer, identify exceptions and arbitrate conflicting corrections. A product can spare them the training infrastructure. It cannot know from an edit alone whether a professional applied a universal rule or accommodated one client’s unusual instruction. Any claim of cheaper customization needs both a compute bill and an expert-time bill.

The competitor sells a different control surface

Trajectory is entering a market that already sells access to model customization. Thinking Machines Lab’s Tinker description says its API handles sampling, gradients, optimizer steps and checkpoints while customers control their data and training loop. That is a more explicit division of labor: the platform runs difficult infrastructure, and technically capable users write the training recipe. Trajectory’s proposed distinction is a product surface that turns live corrections into signal and helps nonresearch teams decide what to change. Both accounts are vendor descriptions. There is no public head-to-head customer trial here establishing one as safer or more effective.

The difference is practical. A research team with its own evaluators may value a fine-tuning API precisely because it leaves the reward and dataset in its hands. A product team whose expertise sits with lawyers or service operators may prefer software that gathers traces and makes a rubric usable. The latter asks more of the vendor’s judgment. It also asks the buyer to trust that the vendor can preserve context when it translates a professional’s correction into a model update. Karanam’s design is a bet that this translation can be made legible enough for specialists to steer.

The site already has adjacent reporting on why legal AI vendors weigh owned models against frontier APIs and on the engineers required after an AI pilot. Trajectory sits where those two problems meet. Owning a specialist model can give a company more control over training; it also gives that company responsibility for evaluation, security and release management. Hiring a forward-deployed team can customize one customer’s workflow, but repeating that labor for every client strains the economics. The public record does not yet show how much of that work Trajectory’s interface removes. Its product is trying to turn bespoke post-training into an ordinary operating process.

A buyer should therefore compare alternatives at the level of a task, not a logo. Put the same held-out contract review, support case or research request through the current system, an engineered prompt or retrieval change, a conventional fine-tuning run and Trajectory’s workflow. Record expert hours, training and inference cost, latency, false approvals, missed requirements and rollback time. Include at least one rare exception that the experts have marked as consequential: a disputed clause, a restricted refund or an outdated source. That is an evaluative proposal based on the competing product claims, not a test Digidai performed. It would reveal whether Karanam’s additional layer earns its place between the customer and the model, and whether the labor saved in setup returns as labor spent reviewing failed edge cases.

Customer data sets the boundary

Karanam was asked about customer-data restrictions during the Sequoia event. He said a system could sample distributions from customer data, generate synthetic examples and compare distributions without directly training on the underlying records. He connected the problem to work he had done at Apple. That answer identifies a possible route, but it is not a published privacy proof for Trajectory’s current platform. Synthetic examples can still reflect private patterns; distribution checks can miss rare sensitive cases; and a customer’s contract may restrict even intermediate processing. The actual protection depends on implementation and agreement.

Trajectory’s homepage says customers decide which data trains and displays a SOC 2 certification claim. Those are useful starting points for procurement. They do not answer which raw traces are retained, whether tool outputs contain privileged information, how tenant isolation works, or whether a model trained for one customer can influence another. A professional edit can expose more than the original task: a lawyer may paste privileged client context, or a support worker may add a customer’s account detail. The company may have controls that are not described publicly. A buyer should ask to inspect them before enabling automatic collection.

The data boundary also intersects with the learning boundary. If a correction is globally useful, a vendor may want to propagate it across customers. If it encodes a private policy, it must stay local. If it is a factual update, a retrieval record may be easier to change and reverse than model weights. Karanam says the system should allocate information among model, harness and context. The public materials have not yet demonstrated a reproducible rule for deciding that allocation. A customer needs a versioned record of where the information went, who approved it and which outputs it changed.

The NIST AI Risk Management Framework core calls for post-deployment monitoring that includes user input, override, incident response, recovery and change management. It is voluntary guidance, not an audit of Trajectory or a claim that its product implements every practice. It frames the operational burden correctly. Continuous learning increases the frequency of meaningful changes. A team needs an accountable release owner even when a model update was proposed by an automated grader and delivered through a polished interface.

A version ledger buyers can inspect

The most persuasive proof of Karanam’s idea would be mundane. Take a task category with a known expert standard. Freeze a representative holdout set before training. Record the old model, prompt, harness and retrieval versions. Collect corrections with named reviewer roles and permissions. Show which corrections went into each layer and which were rejected. Release a new version to a bounded population. Measure whether the error that triggered the change fell without a new error elsewhere. Record adverse cases as well as average scores. Keep a rollback that actually restores the previous behavior, then demonstrate it on the same task category. A version ledger like that would let a buyer distinguish learning from an untracked prompt fix or a change in the work arriving at the agent.

Those steps sound slower than the phrase “learns from every use.” They are what would make that phrase safe enough for a legal, financial or customer-service buyer. A continuous system can still have a review cadence. It can learn from daily use without deploying every proposed change immediately. It can also leave some feedback unlearned when the source is ambiguous, unauthorized or too narrow to generalize. Karanam’s public argument gives the team a way to frame those choices. The November papers give two research tools for reducing the cost of grading and training. The missing public record is an end-to-end, versioned customer result over time.

The stakes are visible in the three kinds of evidence now on the table. TechCrunch’s financing report establishes that investors and several customers see a market. The Trajectory papers report benchmark progress under controlled conditions, while acknowledging overfitting, self-audit and setup limits. Karanam’s own writing sets an ambitious product target: users should shape the intelligence inside their software through the work they already do. None of these should be substituted for the others. A funding round is not an evaluation; a benchmark is not a production SLA; a founder’s design is not an outcome.

A lawyer correcting an omitted clause does not want to become a machine-learning engineer. Nor does that lawyer want the same correction silently applied to an unrelated client. Karanam’s company has built its case around that gap. Its public platform now gives buyers a place to start asking precise questions. The answer will be in the change log after the next correction: what moved, who checked it, and whether the next piece of work was actually better.