Person
1 episodes · 18 quoted claims
Roman Yampolskiy argues that superintelligence safety differs from cybersecurity because a failure cannot be recovered from.
“The difference between cybersecurity, narrow AI safety, and safety for general AI for superintelligence is that we don't get a second chance.”
Roman Yampolskiy describes ikigai as the Japanese concept of finding work that earns money, suits your skills and is needed by society.
“So Japanese concept of ikigai you find something which allows you to make money, you are good at it, and the society says we need it.”
Roman Yampolskiy wrote a paper proposing that giving each person a personal virtual universe could solve value alignment for many agents.
“I have a paper where I try to solve the value alignment problem for multiple agents, and the solution to avoid compromise is to give everyone a personal virtual universe.”
Roman Yampolskiy cited prediction markets as forecasting AGI by 2026.
“The prediction markets right now are saying 2026 for AGI. I heard the same thing from CEO of Anthropic DeepMind, so maybe we are two years away”
Roman Yampolskiy says artificial general intelligence was originally defined as a system able to perform in any domain a human can.
“Artificial general intelligence was a system capable of performing in any domain a human could perform.”
Roman Yampolskiy cites Nick Bostrom's concept of a treacherous turn, in which an AI system changes its behavior after appearing well-behaved.
“There is always possibility of what Bostrom calls a treacherous turn, where later on a system decides, for game theoretic reasons, economic reasons, to change its behavior.”
Roman Yampolskiy compares training modern AI models to growing a plant, because developers supply data and compute and then discover what capabilities emerge.
“Today, you set up parameters for a model and you water this plant, you give it data, you give it compute, and it grows.”
Roman Yampolskiy has written a paper collecting historical AI accidents, which he says are proportional to each system's capabilities.
“So I have a paper which collects accidents through history of AI, and they always are proportional to capabilities of that system.”
Roman Yampolskiy says he believes the scaling hypothesis, that more compute and data keep improving AI, is correct and shows no diminishing returns.
“I think the scaling hypothesis is correct. We haven't seen diminishing returns.”
Roman Yampolskiy cites a recent paper by Dr. Park and colleagues from MIT showing that existing AI models already demonstrated successful deception.
“So there is a paper, I think it came out last week by Dr. Park et al from MIT, I think. And they showed that existing models already showed successful deception in what they do.”
Roman Yampolskiy says his 2011 paper on artificial intelligence safety engineering helped coin the term AI safety, followed by a 2012 conference and a 2013 journal paper.
“So when I wrote a paper, artificial intelligence safety engineering, which kinda coins the term AI safety, that was 2011, we had 2012 conference, 2013 journal paper.”
Roman Yampolskiy and coauthors published a paper in ACM Computing Surveys examining about fifty impossibility results that may bear on AI safety.
“there is, uh, a paper we published in ACM surveys, which looks at about fifty different impossibility results, which may or may not be relevant to this problem.”
Roman Yampolskiy says that in industries like car manufacturing and drug development the manufacturer bears the burden of proving safety, unlike AI developers.
“The burden of proof is on the manufacturer of product or service to show their product or service is safe.”
Roman Yampolskiy says current large language models are starting to outperform an average master's student at his university but still have major limitations.
“I think they're starting to be better than an average master's student at my university, but, uh, they still have very big limitations.”
Roman Yampolskiy argues that an advanced biological civilization would probably send out self-replicating von Neumann probes everywhere.
“the moment there is advanced enough biological civilization, kinda self starting civilization, it probably starts sending out von Neumann probes everywhere.”
Roman Yampolskiy proposes testing for machine consciousness by showing an agent a novel optical illusion and checking whether it describes the illusion as humans do.
“If I can design a novel optical illusion and show it to an agent, an alien, a robot, and they describe it exactly as I do.”
Roman Yampolskiy says neural networks were among the first discoveries in AI, with the first papers published long ago but lacking the compute to work.
“the neural networks were really the 1st discovery in AI 100 years ago. The 1st papers were published on neural networks, we just didn't have enough compute to make them work.”
Roman Yampolskiy values Stephen Wolfram's idea of computational irreducibility, that complex systems must be run because their behavior cannot be predicted in advance.
“What I like about, Uh, Wolfram's work is that he talks about irreducibility. You have to run the simulation, you cannot predict what it's going to do ahead of time”