2022
The company had only eight people when it pivoted to building a coding product.
“So we were only eight people at the time.”
Topic
Codeium
16 facts from 1 episode, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.
2022
Codeium shipped its first product and posted it on Hacker News within about six weeks of starting work.
“within a month and a half, we shipped a product and put it on HN.”
2022
Codeium initially used Salesforce's CodeGen model to get its product running before training its own.
“And we originally, actually, to get a product up and running, we used Salesforce's CodeGen product.”
2023
Codeium grew from fewer than 1,000 users at the start of 2023 to over 500,000 users by the end of 2023.
“in the beginning of 2023, we had less than a thousand users, and by the end of 2023, we had over 500,000 users that used the product.”
Undated
Exafunction, the predecessor of Codeium, managed more than 10,000 GPUs in the public cloud for a few autonomous vehicle customers.
“We were managing upwards of 10,000 GPUs in the public cloud for a handful of autonomous vehicle customers.”
Undated
Varun Mohan says that when developers use Codeium, over 45% of their software is generated by it.
“and when developers use Codeium, over 45% of their software is actually generated by Codeium.”
Undated
Codeium found that using context from other parts of a codebase raised the amount of autocomplete code accepted by 27%.
“by leveraging context for even something like autocomplete, even outside of chat, we we found 27% uptick in amount of code accepted.”
Undated
Codeium found GPT-4 more steerable than GPT-3.5 for code edits, but its per-token latency was too high, so it trained its own edit model.
“And then what we did instead was we later used GPT four because it it actually is more steerable, but then the latency on a per token basis is too high for users to want to use the product.”
Undated
Varun Mohan says a customer dataset of 100,000 to a million lines of code is only about 10 million tokens, tiny next to the trillions used in pre-training.
“training on a S on a on a data set that is like, let's say, 100,000 or a million lines of code is only 10 million tokens.”
Undated
Codeium observed a large drop in usage when autocomplete latency increased by a couple hundred milliseconds.
“If we add a couple hundred milliseconds of latency to autocomplete, we've noticed a a massive decline in sort of the usage of the product”
Undated
Varun Mohan says some of Codeium's largest customers measure percentage of code written by the tool at roughly 40 to 50%.
“We've worked with some of the largest companies and they're measuring the percentage code written to be somewhere in the forty to fifty range.”
Undated
Varun Mohan says Codeium receives over a million pieces of user feedback each hour, used to judge product quality rather than to train generative models.
“every hour we get mil, we get over a million pieces of feedback from our users on is our product getting better.”
Undated
Varun Mohan says GitHub Copilot published a post claiming it did not output security vulnerabilities, which Codeium says it could reproduce.
“one of our competitors, obviously the biggest one, Copilot, came out with a post where they were like, Hey, we don't output security vulnerabilities.”
Undated
Codeium tries to release new models about once per quarter.
“we generally speaking, try to roll out new models every quarter or so.”
Undated
Codeium's evaluation uses open source code it does not train on, strips functions that unit tests cover, and checks whether the model can restore code that passes the tests.
“We take a bunch of open source code we fi that we don't train on.”
Undated
Codeium runs its code evaluations in gVisor sandbox containers so generated code cannot damage a machine, such as by running rm -rf.
“And we run those in in sort of gVisor containers because the goal is like, we don't want this thing to rm rf, right?”
Episodes