In brief
Zach Lloyd of Warp argues that a "software factory" should be a concrete, measurable system, not a loose pile of AI automations. The factory is defined in code, runs in the cloud and is fed work through tools like Slack and Linear. Because every agent conversation is stored, a team can measure cost and quality and tune the system the way it would any other engineering project.
The main tension is between rigor and overhead. Lloyd says teams should stop judging coding agents by Twitter hype and measure them with scorers, benchmarks and self-improvement loops. He admits this is overkill for very small teams, which are better off on cheap subscriptions. It is also a poor fit for giants with bespoke infrastructure, and he describes the middle band of roughly 10 to 100 engineers as the sweet spot.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Zach Lloyd says Warp's cost per pull request was about $80 when it first adopted the factory approach, partly because it used Opus where it was not needed.
“when we first started using the factory approach, it was costing us about eighty per PR because we were using, like, Opus, where we shouldn't have.”
0:23:39 · Zach Lloyd · Cost per PR
Zach Lloyd says Warp tuned its factory until cost per pull request fell to roughly $15 to $20.
“we were able to get the cost per PR down to, like, twenty bucks or fifteen bucks, something like that.”
0:24:14 · Zach Lloyd · Cost reduction
Zach Lloyd says Warp's pull request throughput rose by about 40 after tuning its factory over the previous month.
“we can measurably improve the throughput of our PRs over the last month. It's gone up by forty.”
0:24:14 · Zach Lloyd · Throughput
Warp defines each software factory in a YAML file that lists the repositories the agents can use, along with integrations and workflow.
“So for us, we basically have a YAML file which says, okay, these are all the repos that you can be using.”
0:05:11 · Zach Lloyd · Factory as code
In Warp's model, a company has one software factory for each product surface, such as a mobile app or server app.
“You can orchestrate it, you can measure it. You have one factory per product surface,”
0:04:15 · Zach Lloyd · Factory design
Zach Lloyd says about 60% of Warp's internal factory work starts in Slack.
“the most obvious one is Slack, and, like, we're doing, like, 60% of our work is starting in Slack.”
0:25:15 · Zach Lloyd · Slack usage
Zach Lloyd says Warp uses Namespace as its primary agent sandbox provider for compute.
“We use namespace as our primary one.”
0:22:26 · Zach Lloyd · Sandbox provider
Zach Lloyd says Warp tests iOS apps by running a Mac virtual machine where an agent operates the simulator.
“the way that we test, like iOS apps, is you're gonna wanna have the ability to run a MacVM and actually, like, have it run the simulator and click around.”
0:22:40 · Zach Lloyd · Computer use
Zach Lloyd says Warp factories let customers plug in their own agent harness, including Pi or any ACP-compatible harness.
“You can even plug in your own harness, which I think is a pretty big deal. So, you know, you you can orchestrate things like Pi, like I mentioned earlier, and basically any ACP compatible harness.”
0:23:39 · Zach Lloyd · Harness support
In an example benchmark shown by Zach Lloyd, GPT 5-6 Sol was 95% correct at about $6 per task while GLM 5-3 flash was 93% correct at about $2.
“GPT 5-6 Sol here was, like, 95% correct on this task set, but it costs six bucks on average, and GLM 5-3 flash was 93% and cost two bucks.”
0:17:31 · Zach Lloyd · Model benchmark
Zach Lloyd says factory data can track traditional DORA metrics, including how long it takes to ship something.
“things like, you know, traditional DORA metrics, how long does it take to ship something?”
0:08:24 · Zach Lloyd · DORA metrics
Zach Lloyd says Warp's evals work as an LLM-as-judge setup in which a chosen judge model grades stored agent runs.
“the eval is, again, think of it as LLM as a judge. So you're picking a a judge model.”
0:11:14 · Zach Lloyd · Evals
Zach Lloyd says self-improvement agents can open pull requests against Warp's factory definition to improve the factory.
“because you have your whole factory defined as code, and these are coding agents, they can essentially make a pull request against the factory definition to try to improve the factory itself.”
0:15:21 · Zach Lloyd · Self-improvement
Zach Lloyd gives an example in which Warp's self-improvement agents found 54 triage failures that a sub-agent could have avoided.
“we have fifty four failure cases of this particular form, where it would've been better if our triager had been able to spawn a sub-agent to do its investigation.”
0:14:51 · Zach Lloyd · Triage failures
Zach Lloyd compares defining a software factory as code to using Terraform for infrastructure.
“So, again, the analogy is, like, Terraform here, you get the same benefits, but actually more.”
0:19:51 · Zach Lloyd · Infrastructure as code
Zach Lloyd says larger companies want different MCP access for different departments such as marketing, sales and engineering.
“So they want things like, what MCPs can the marketing team access versus the sales team versus the engineering team?”
0:22:00 · Zach Lloyd · Access control
Zach Lloyd says a software factory starts to make sense at roughly five to ten engineers.
“We start seeing that maybe around, like, f- five to ten engineers would maybe be the floor where this makes sense.”
0:27:41 · Zach Lloyd · Team size
Zach Lloyd names Uber and Ramp as examples of companies that are a poor fit for Warp factories because they build their own systems.
“our our kind of not a good fit customer for this, for instance, would be like Uber or Ramp.”
0:30:34 · Zach Lloyd · Customer fit
The host says a $200 coding subscription plan provides roughly $8,000 worth of API usage.
“you get, uh, for, like, 200 plan, you get 8,000 of API, right?”
0:32:10 · swyx · Subscription subsidy
Zach Lloyd says Cursor made a good exit by becoming a model lab after recognizing the economics of building on third-party models.
“I mean, Cursor was brilliant, they did it, and then I think that they had a perfect exit and became a model lab because I think they fully realized the economics of it.”
0:32:35 · Zach Lloyd · Cursor strategy
Zach Lloyd says factory tooling is used mainly by engineering managers, CTOs and developer-experience or platform teams, not day-to-day engineers.
“they'll often have an internal team that's been trying to, like, it's like the DevEx team or the platform team”
0:12:29 · Zach Lloyd · Target users
The host says AIE Code is the last AIE flagship event of the year and will be held in San Francisco.
“AIE Code is the last, uh, AIE flagship of the year. It's gonna be in San Francisco, where we're gathering all the top agentic engineers”
0:00:54 · swyx · AIE Code