Niro Rajadurai says that XBOW's founder, Hugo Demoor, was the person who invented GitHub Copilot.
“the founder of XBOW, Hugo Demoor was actually the person that invented GitHub Copilot.”
Person
Niro Rajadurai
1 episodes · 22 quoted claims
Rajadurai says he ran the advanced security business at GitHub, where an early use of AI models was fixing vulnerabilities already identified in existing code.
“because I ran the advanced security business at GitHub, was can we actually use these models to help us remediate vulnerabilities that have been identified in existing code?”
Rajadurai says that in the largest enterprises the volume of code is still bigger than what an AI model's context window can handle.
“the volume of code that the models need to deal with are still bigger than what the models context window can cope with.”
Rajadurai defines a vulnerability as exploitable if one can build something that uses it to steal information or run commands on a target machine.
“Because exploitable means I can build something that can use that vulnerability to go and exfiltrate some information or go and run some special command on that target machine”
Rajadurai says some open-source AI models now perform as well as the frontier models that providers are trying to restrict.
“some of the open source models are actually as performant as these models that we're trying to gate to the broader community.”
Rajadurai says Anthropic's models are currently being gated to prevent access by the broader world.
“I think like, what's really scary now is that we all know what's happening with the Anthropic models right now, where they're being gated and and prevent access to the broader world.”
Rajadurai argues that an attacker needs to find only one hole in an application, while a defender must find all of them faster.
“if I have an application and the application has four holes, the attacker only needs to find one of those holes.”
Rajadurai says some of the largest enterprises have roughly 16,000 to 20,000 public-facing applications.
“a lot of them have a big public facing estate for their application, maybe sixteen, 20,000, you know, in the some of the largest enterprises, public facing application.”
Rajadurai says companies typically run penetration tests only once or at best twice a year on an application, usually the riskiest ones, because they are slow and costly.
“But they're only doing this like once a year, or at best twice a year on an application.”
Rajadurai says penetration testing is the gold standard for validating an application, which is why regulators ask whether it has been done.
“penetration testing activity is kind of the gold standard of like validating an application, which is why regulators ask you, have you pen tested your application?”
Rajadurai says an AI model that finds a SQL injection may try to drop database tables, unlike an experienced human hacker who would avoid wiping a production system.
“It turns out if you use a model to do the same exercise, the model say, Oh wow, I found a SQL injection, I'm gonna do a drop table.”
Rajadurai says AI models are very good at suggesting fixes when given a clear description of the problem, how to exploit it and what the exploit produces.
“It turns out that if you give a really good description of the problem, how to go and exploit the problem and what the outcome is if you run that exploit.”
Rajadurai describes a customer case where a web application firewall rule blocked a URL pattern, but XBOW bypassed it by encoding a letter as an ASCII code with a percent sign.
“But in programming, you can represent letters as they actually are, or you can represent them as an ASCII code by putting a percent there.”
Rajadurai says XBOW found that pairing different AI models in one reasoning process gives better results than a single model, an approach it calls alloy models.
“pairing different models together as part of a reasoning process actually results in higher quality and better results than just leveraging a single model.”
Rajadurai says XBOW does not train its own models but uses existing ones with coaching and guidance provided in the context window.
“we actually don't try to to train the model, we we leverage the models as they are”
Rajadurai says XBOW is two and a half years old and announced that it became number one on HackerOne in July of the previous year.
“XBOW's two and a half years old. We did our big, like, you know, announcement, Hacker One, where we became number one back in July last year”
Rajadurai says AI models tend to try to please users, which gives LLM-based vulnerability scanners a high false positive rate and a lot of noise.
“they tend to want to keep you happy. And so they tend to have a high false positive rate.”
Rajadurai estimates that 95% of autonomous offensive security vendors tell the market that a human must be in the loop.
“I think 95% of the players out there are telling the market, you have to have a human in the loop to use these tools.”
Rajadurai says XBOW was designed from the start to be fully autonomous, given no prior information and needing no human in the loop.
“that XBOW would be a fully autonomous tool that would be given zero information and would require zero humans in the loop.”
Rajadurai says the average time to exploit a known vulnerability was about 1.3 years in 2020.
“the time to exploit, like going back back to 2020 was about 1.3 years”
Rajadurai says time to exploit has fallen to about one day and is predicted to fall to one minute by the end of 2027.
“It's now down to like one day, and they're predicting that by the end of 2027, it'll be down to one minute.”
Rajadurai says Anthropic's 'Mythos' model has led boards to ask security teams whether they are ready for an AI-powered attack.
“I think Anthropic have done a wonderful job, which is like, you know, are you mythos ready, right?”