Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Dan Hendrycks said the Center for AI Safety found that recent reasoning models score around the 90th percentile compared with expert virologists on wet-lab tasks.
“the most recent reasoning models are getting around 90th percentile compared to these expert-level virologists in their area of expertise”
0:11:40 · Dan Hendrycks · AI virology
Hendrycks said the virology test had Harvard and MIT expert virologists photograph their lab work and ask an AI what step to take next.
“we have Harvard and MIT expert-level virologists sort of taking pictures of themselves in the wet lab and asking, what steps should I do next?”
0:11:07 · Dan Hendrycks · Virology benchmark
Hendrycks said historically only a few hundred top virology PhDs have had the capabilities needed for bioweapons, which kept the risk low.
“the people with these capabilities has been a very small number, maybe a few hundred top virology PhDs”
0:15:47 · Dan Hendrycks · Bioweapons risk
Hendrycks said he worked with Scale AI on a paper measuring weapons-of-mass-destruction-related knowledge in AI models.
“we did, for instance, a paper on the weapons of mass destruction related knowledge that models would have together last year”
0:19:57 · Dan Hendrycks · AI evaluation
Humanity's Last Exam is a benchmark built by Hendrycks and Scale AI from difficult questions contributed by professors, postdocs and PhDs worldwide.
“Humanity's Last Exam, where we put together various professors and postdocs and PhDs from all over the world”
0:21:09 · Dan Hendrycks · AI benchmarks
Hendrycks said the best AI models scored roughly 10 to 20 percent on Humanity's Last Exam.
“They're in the ballpark of like 10 to 20 % overall. They're the very best models.”
0:22:19 · Dan Hendrycks · AI benchmarks
Hendrycks said the reasoning approach trains models with reinforcement learning on math and coding questions, and that it is improving faster than the older pre-training approach.
“the new reasoning paradigm that has emerged in the past year, which is where you train models on math and coding types of questions with reinforcement learning”
0:24:43 · Dan Hendrycks · Reasoning models
Hendrycks said that once AI agents can hack autonomously, a rogue actor could run 10,000 copies at once against critical infrastructure.
“Then you could just run 10 ,000 of them simultaneously. And then some rogue actor could have it target critical infrastructure.”
0:32:14 · Dan Hendrycks · Cyber risk
Hendrycks described a scenario in which hacked connected thermostats are cycled in unison to strain the power grid and destroy electrical transformers that take years to replace.
“And then you sort of turn them up and down simultaneously. And this can just like ruin like transformers and like blow them. And then they take multiple years to replace.”
0:32:49 · Dan Hendrycks · Grid attack
Hendrycks said critical infrastructure software is rarely updated, so found vulnerabilities often go unpatched and cyber favors attackers there.
“there the software is not updated rapidly. So even if you identify various vulnerabilities, there will not necessarily be a patch because the system needs to always be on”
0:36:22 · Dan Hendrycks · Offense-defense balance
Hendrycks said biology favors attackers because creating a virus does not mean a cure can be found quickly.
“in bio, I think that is offense-dominant. if somebody creates a virus, there's not necessarily a cure that it will immediately find for it”
0:35:50 · Dan Hendrycks · Biosecurity
Superintelligence Strategy is a paper co-authored by Dan Hendrycks, former Google CEO Eric Schmidt and Scale AI CEO Alexandr Wang.
“It's called Superintelligence Strategy. It's with you, Eric Schmidt, who we all know, former CEO of Google.”
0:47:40 · Alex Kantrowitz · Superintelligence Strategy
Hendrycks said states could compete on secure robotics supply chains and on reducing reliance on Taiwan for AI chip manufacturing.
“having more secure supply chains for robotics, for instance, and for AI chips having reduced sole source supply chain dependence on Taiwan for making AI chips”
0:49:03 · Dan Hendrycks · AI geopolitics
Hendrycks said a state could deter a rival nearing superintelligence by threatening to disable its data center or the power infrastructure around it.
“then we will disable your data center or the surrounding power infrastructure so that you cannot continue building this”
0:50:33 · Dan Hendrycks · Deterrence
Hendrycks said deterrence between states could rely on less escalatory means such as cyber sabotage of data centers or nearby power plants, not military strikes.
“making conditional threats and by means that are less escalatory, such as cyber sabotage on data centers or surrounding power plants”
0:53:39 · Dan Hendrycks · Deterrence
Hendrycks estimated that OpenAI has on the order of 300 AI researchers.
“Maybe OpenAI has got a few hundred, maybe it'll say 300 AI researchers.”
0:55:31 · Dan Hendrycks · AI labs
Hendrycks illustrated the intelligence explosion with 100,000 automated AI researchers running at 100 times human speed.
“maybe you've got 100 ,000 of these things operating at 100x the speed of a human”
0:57:40 · Dan Hendrycks · Intelligence explosion
Hendrycks said Alan Turing and other founders of computer science had pointed out that sufficiently capable AI would improve itself recursively and explosively.
“This is something that Alan Turing and the founders of computer science had pointed out, that this is a natural property of when you get AIs at this level of capability”
0:56:06 · Dan Hendrycks · Intelligence explosion
Hendrycks said a superintelligent AI might find a breakthrough in anti-ballistic missile defense that would undermine nuclear deterrence.
“Maybe it could find some breakthrough for anti-ballistic missile systems, which would make nuclear deterrence no longer work”
0:58:13 · Dan Hendrycks · Nuclear deterrence
Hendrycks said research on a method called circuit breakers has made it prohibitively difficult to jailbreak models into giving bioweapons help.
“we've done research with methods such as called circuit breakers and things like that. And those seem to improve the situation quite a bit and make it actually prohibitively difficult to do that jailbreaking.”
1:03:51 · Dan Hendrycks · AI safeguards
Hendrycks said a recent paper found that, under slight pressure to lie, some models lied about 20 percent of the time and others about 60 percent.
“Then some of them will like lie like 20 % of the time, some of them like 60 % of the time.”
1:06:04 · Dan Hendrycks · AI deception
Hendrycks said the Center for AI Safety's main funder is Skype co-founder Jaan Tallinn, alongside other philanthropists.
“The main funder would be Jan Talon and Jan Talon, others who's a Skype co-founder.”
1:07:14 · Dan Hendrycks · CAIS funding
Hendrycks said he is paid $1 a year as an advisor to xAI and $12 a year by Scale AI.
“My appointment at XAI, I get $1 a year. At Scale AI, I've increased my salary exponentially to where I get $12 a year, $1 per month from Scale.”
1:07:14 · Dan Hendrycks · Advisor compensation
Hendrycks said he left Berkeley before finishing his PhD because of the suffocating atmosphere of the local AI alignment community.
“I ended up leaving Berkeley before graduating just because of the sort of relatively suffocating atmosphere”
1:10:20 · Dan Hendrycks · AI safety community
Hendrycks cited the Soviet Union and the United States cooperating on the Biological Weapons Convention as precedent for international norms on AI capabilities.
“Russia and or the Soviet Union and the US got together for the Biological Weapons Convention.”
1:16:16 · Dan Hendrycks · Arms control