Undated
Reganti says that after about three months, underwriters at a bank using an AI loan-review assistant began asking deeper, unanticipated questions, requiring the system to be rebuilt in part.
“post three months, we realized that they were so excited with the product that they started asking very deep questions that we never anticipated.”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 1:00:18
Undated
Reganti says a fully autonomous customer-support agent she worked on was shut down because the team could not keep up with the hot fixes for emerging problems.
“we had to shut down the product because we were doing so many hot fixes and there was no way we could, um, count all the emerging or, uh, emerging problems”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:46:53
Undated
Reganti estimates that replacing a critical workflow or reaching significant ROI with AI takes four to six months of work, even with a strong data and infrastructure foundation.
“To replace any critical workflow or to, um, build something that can give you significant ROI, easily takes four to six months of work, even if you have the best data layer and infrastructure layer.”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:31:46
Undated
Reganti says retail companies' product category hierarchies are often messy, which makes even the simple task of routing support tickets hard for an agent.
“Any retail company, any popular retail company that you can think of, has hierarchical taxonomies. Most of the times, the taxonomies are incredibly messy.”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:51:39
Undated
Reganti describes insurance pre-authorization of items like blood tests and MRIs as a good AI use case, while invasive surgery should keep a human in the loop.
“Insurance pre authorization is a very ripe use case for AI because, uh, clinicians spend a lot of time, um, pre authorizing, uh, things like blood tests, MRIs and things like that, right?”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:16:34
Undated
In the continuous development half of the guests' framework, a team scopes the capability and curates a dataset of expected inputs and outputs before building.
“we have this right side of the loop, which is continuous development. Uh, where you scope capability and curate data, essentially get a data set of what your expected inputs are”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:47:52
Undated
Lenny Rachitsky introduces Aishwarya Reganti as an early AI researcher at Alexa and Microsoft with over 35 published research papers.
“Ash was an early AI researcher at Alexa and Microsoft and has published over 35 research papers.”
Lenny Rachitsky · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 0:01:29
Undated
Reganti says she and Badam maintain a free GitHub repository of AI learning resources with about 20,000 stars.
“We also have a GitHub repository with about 20K stars, and that repository is all about good resources for learning AI.”
Aishwarya Naresh Reganti · Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon, 1:24:41