Person
Dan Hendrycks
This page collects what Dan Hendrycks said on the Big Technology Podcast about the rising risks of advanced AI, including hacking, virology and loss of control. Each point is attributed to him and is his own view or claim, not established fact.
Hendrycks said the Center for AI Safety found recent reasoning models score around the 90th percentile against expert virologists on wet-lab tasks. He also described how autonomous AI hacking could threaten critical infrastructure, and said states might deter a rival nearing superintelligence through cyber sabotage of data centers. He told the host that he left Berkeley before finishing his PhD.
1 episodes · 24 quoted claims
Dan Hendrycks said the Center for AI Safety found that recent reasoning models score around the 90th percentile compared with expert virologists on wet-lab tasks.
“the most recent reasoning models are getting around 90th percentile compared to these expert-level virologists in their area of expertise”
Big Technology Podcast · 0:11:40
Hendrycks said the virology test had Harvard and MIT expert virologists photograph their lab work and ask an AI what step to take next.
“we have Harvard and MIT expert-level virologists sort of taking pictures of themselves in the wet lab and asking, what steps should I do next?”
Big Technology Podcast · 0:11:07
Hendrycks said historically only a few hundred top virology PhDs have had the capabilities needed for bioweapons, which kept the risk low.
“the people with these capabilities has been a very small number, maybe a few hundred top virology PhDs”
Big Technology Podcast · 0:15:47
Hendrycks said he worked with Scale AI on a paper measuring weapons-of-mass-destruction-related knowledge in AI models.
“we did, for instance, a paper on the weapons of mass destruction related knowledge that models would have together last year”
Big Technology Podcast · 0:19:57
Humanity's Last Exam is a benchmark built by Hendrycks and Scale AI from difficult questions contributed by professors, postdocs and PhDs worldwide.
“Humanity's Last Exam, where we put together various professors and postdocs and PhDs from all over the world”
Big Technology Podcast · 0:21:09
Hendrycks said the best AI models scored roughly 10 to 20 percent on Humanity's Last Exam.
“They're in the ballpark of like 10 to 20 % overall. They're the very best models.”
Big Technology Podcast · 0:22:19
Hendrycks said the reasoning approach trains models with reinforcement learning on math and coding questions, and that it is improving faster than the older pre-training approach.
“the new reasoning paradigm that has emerged in the past year, which is where you train models on math and coding types of questions with reinforcement learning”
Big Technology Podcast · 0:24:43
Hendrycks said that once AI agents can hack autonomously, a rogue actor could run 10,000 copies at once against critical infrastructure.
“Then you could just run 10 ,000 of them simultaneously. And then some rogue actor could have it target critical infrastructure.”
Big Technology Podcast · 0:32:14
Hendrycks described a scenario in which hacked connected thermostats are cycled in unison to strain the power grid and destroy electrical transformers that take years to replace.
“And then you sort of turn them up and down simultaneously. And this can just like ruin like transformers and like blow them. And then they take multiple years to replace.”
Big Technology Podcast · 0:32:49
Hendrycks said critical infrastructure software is rarely updated, so found vulnerabilities often go unpatched and cyber favors attackers there.
“there the software is not updated rapidly. So even if you identify various vulnerabilities, there will not necessarily be a patch because the system needs to always be on”
Big Technology Podcast · 0:36:22
Hendrycks said biology favors attackers because creating a virus does not mean a cure can be found quickly.
“in bio, I think that is offense-dominant. if somebody creates a virus, there's not necessarily a cure that it will immediately find for it”
Big Technology Podcast · 0:35:50
Hendrycks said states could compete on secure robotics supply chains and on reducing reliance on Taiwan for AI chip manufacturing.
“having more secure supply chains for robotics, for instance, and for AI chips having reduced sole source supply chain dependence on Taiwan for making AI chips”
Big Technology Podcast · 0:49:03
Hendrycks said a state could deter a rival nearing superintelligence by threatening to disable its data center or the power infrastructure around it.
“then we will disable your data center or the surrounding power infrastructure so that you cannot continue building this”
Big Technology Podcast · 0:50:33
Hendrycks said deterrence between states could rely on less escalatory means such as cyber sabotage of data centers or nearby power plants, not military strikes.
“making conditional threats and by means that are less escalatory, such as cyber sabotage on data centers or surrounding power plants”
Big Technology Podcast · 0:53:39
Hendrycks estimated that OpenAI has on the order of 300 AI researchers.
“Maybe OpenAI has got a few hundred, maybe it'll say 300 AI researchers.”
Big Technology Podcast · 0:55:31
Hendrycks illustrated the intelligence explosion with 100,000 automated AI researchers running at 100 times human speed.
“maybe you've got 100 ,000 of these things operating at 100x the speed of a human”
Big Technology Podcast · 0:57:40
Hendrycks said Alan Turing and other founders of computer science had pointed out that sufficiently capable AI would improve itself recursively and explosively.
“This is something that Alan Turing and the founders of computer science had pointed out, that this is a natural property of when you get AIs at this level of capability”
Big Technology Podcast · 0:56:06
Hendrycks said a superintelligent AI might find a breakthrough in anti-ballistic missile defense that would undermine nuclear deterrence.
“Maybe it could find some breakthrough for anti-ballistic missile systems, which would make nuclear deterrence no longer work”
Big Technology Podcast · 0:58:13
Hendrycks said research on a method called circuit breakers has made it prohibitively difficult to jailbreak models into giving bioweapons help.
“we've done research with methods such as called circuit breakers and things like that. And those seem to improve the situation quite a bit and make it actually prohibitively difficult to do that jailbreaking.”
Big Technology Podcast · 1:03:51
Hendrycks said a recent paper found that, under slight pressure to lie, some models lied about 20 percent of the time and others about 60 percent.
“Then some of them will like lie like 20 % of the time, some of them like 60 % of the time.”
Big Technology Podcast · 1:06:04
Hendrycks said the Center for AI Safety's main funder is Skype co-founder Jaan Tallinn, alongside other philanthropists.
“The main funder would be Jan Talon and Jan Talon, others who's a Skype co-founder.”
Big Technology Podcast · 1:07:14
Hendrycks said he is paid $1 a year as an advisor to xAI and $12 a year by Scale AI.
“My appointment at XAI, I get $1 a year. At Scale AI, I've increased my salary exponentially to where I get $12 a year, $1 per month from Scale.”
Big Technology Podcast · 1:07:14
Hendrycks said he left Berkeley before finishing his PhD because of the suffocating atmosphere of the local AI alignment community.
“I ended up leaving Berkeley before graduating just because of the sort of relatively suffocating atmosphere”
Big Technology Podcast · 1:10:20
Hendrycks cited the Soviet Union and the United States cooperating on the Biological Weapons Convention as precedent for international norms on AI capabilities.
“Russia and or the Soviet Union and the US got together for the Biological Weapons Convention.”
Big Technology Podcast · 1:16:16