Never Too Much AI: Upwork's Andrew Rabinovich

Stay tuned after today's episode as Shervin and I talk about the changing role of the CIO.

With AI-infused products and services advancing so rapidly, what one component is holding developments back? Find out on today's episode.

I'm Andrew Rabinovich from Upwork, and you're listening to me, myself and AI.

Welcome to Me, myself and AI, a podcast on artificial intelligence and business. Each episode we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Today, Shervin, I are talking with Andrew Rabinovich, vice president and head of AI and machine learning at Upwork. Andrew, thanks for talking with us.

Thank you. Good morning. My pleasure to be here.

So, Andrew, I've long been interested in online labor platforms and Upwork in particular, both as a user, I admit, and as a researcher. In case any listeners are looking for some light bedtime reading, I've recently published an academic paper using online platforms. And what's fascinating is that these IT improvements are shrinking transaction costs, and that's changing structure of firms. I don't know that the economists from 1937 would recognize firms these days. But before I digress too much, let's step back.

Andrew, wrote me in. Tell people what Upwork is and what the platform does.

Sure, so Upwork is a world's largest bidirectional marketplace, which is an important distinction that we can dive into later. Where clients can look for talent across a variety of categories to help them work, and freelancers from around the world can look for work opportunities in online software, any kind of development that allows them to leverage their skills without necessarily being physically close to where the action is.

That makes sense. And you're the vice president and head of AI and machine learning. Tell us about what that role means.

Upwork is a company that, as a result of a merger of two large marketplaces, oDesk and Elance, started in early 2000s and then 2015 merged to become Upwork. And for the longest time, the platform has offered a matching service between clients and freelancers, as I had mentioned. However, in the evolution of technology that you referred to, we are on the path to transform Upwork, where the goal is to distance ourselves away from matching client to talent.

But rather than doing that, offer a outcome driven business where clients come onto the platform and describe the problem that they wanna solve, rather than looking for folks to help them solve the particular problem. And in order to facilitate that experience, there's a tremendous amount of AI and machine learning involved to create an effective and pleasant experience.

So explain that more, you know, I'm a client, I show up and I say I have this technical problem. I'd like to develop a forecasting model for my, uh, my supply Podcast listeners chain. Yeah, for my podcast listeners, actually, yeah, that's a better one, right?

That's a great example. Traditionally, say, you come onto the platform and you say I am looking for a financial analyst to help me improve traction of my podcast. And then you would have to write super technical specifications of the type of a person you're looking for. And then Upwork's match engine would find you a list of highly skilled candidates that you can pick from, and then Upwork steps away and then you transact with that individual, whichever

We're all familiar with that model, right? So the new one is

there's two problems with this original approach, and they're not really problems, they're more limitations. First of all, you have to know what kind of a person you need or or group of people you need to solve the particular task. You have to be able to describe it in very technical terms, such that the search engine on Upwork is able to provide you with the right matches. Now, suppose you are doing a podcast on travel or mycology or design, and you really aren't a technical person, and you're like, I have some traction, people listen to my podcast, but I want it to, like, explode, so I can become an influencer, make $1 million, whatever.

And that's your level of technical expertise. This is as far as you can explain it. You're like, super experienced in your domain, but when it comes to unit economics of podcasting, you have no clue. So essentially, Upwork is useless to you because the barrier to entry into this platform is so high that you can't use it.

you need, you need technical know-how, and you need to know what you need. Yep,

So the expanded direction that we're in right now is we've launched this AI companion on Upwork's platform called Uma, which stands for Upwork's Mindful AI. Where you come to the platform and you say exactly what I just mentioned. I'm a podcaster in domain XYZ. I want my podcast to be better, period. And then Uma figures out that you need an analyst, or you need some technical person to help you expand it, maybe market it elsewhere, maybe get some advertising going, whatever it is that's necessary.

And Uma finds the right people and or AIs that will help you solve your problem, and together will deliver an outcome, rather than just finding you the people that you can work with, right? So rather than finding talent that can help you solve the problem, Upwork now solves the problem.

so clearly there is a generative AI component to that. What else is there, for example, are there recommender engines and forecasting models and things like that as well?

everything, right? So whatever AI technologies exist today, and there's

Off-the-shelf or proprietary?

off-the-shelf. Think of the freelancers that work in the Upwork marketplace or Upwork platform. They're not Upwork employees, right? They're freelancers. In the same way, all existing AI tools, apps in the app store, or components in AWS, they're also AI freelancers, if you will, that should have the right to exist on this platform. And the role of Upwork is to figure out how to combine the human talent with machine talent to deliver the desired outcome.

I am a business school professor, and I have to think about the ROI here. How does that play out in the future for you?

That's a great question, and there is a nuance. You know this a lot better than I do. Just like with any step function in technological progress, you know, electricity, steam engines, whatever, the technology's ability is not to replace people, but rather to amplify them. So if in the past you were able to translate a podcast from English to Japanese for $10,000, tomorrow you'll be able to take the podcast and translate them to all languages in the world for the same $10,000.

So it's not that the high-paying jobs are going away, they're just becoming much, much more interesting. If in the past, for $50,000 on Upwork, you could build an e-commerce website, in the future, for $50,000, you'd be able to build a new Facebook, right? And in fact, this is super promising and makes me optimistic about the future. Because with essentially the same amount of people, you'll be able to create much, much more value.

Because the technology that's becoming available in abundance is only helping you solving larger and larger problems.

Yeah, I think that's a great way to. I mean, there's two things that makes me think of. One is we weren't spending, you know, sorry to our Japanese audience. We were not spending $10,000 to translate each podcast episode. And so, in some sense, your competition there is not with the 10,000 job, it's with us not doing that job at all in the first place. And so there's a big underserved market that has the potential to tap.

And that's in addition to the other point you mentioned, which is okay, we're gonna expand and we can do a lot more languages. I mean, there's just a lot of things that that aren't happening right now because of that barrier, and I think that's what seems very promising about this. Well, you got a complicated people wouldn't know from talking to you because you haven't said much super technical, but you got a crazy technical background, great technical depth.

Tell us a bit about how you ended up in this position.

So it goes back quite a while back. I started doing AI, as we call it today, in the past, we called it machine learning, and in particular, computer vision is where my career began. I got really interested in doing biomedical imaging, and in 2003, I founded the first company that was detecting cancer in human tissue. I didn't know anything about startups, I didn't know really anything about business. The company got acquired for some not interesting amount of money, but I got really, really fascinated with machine learning, and I thought that was the path for me forward.

And then after I got a PhD. in machine learning at UCSD, I joined Google, and together with a bunch of colleagues, we released the world's first AR app for both Android and iOS. That was called Google Goggles. Today, that's the lens feature in the Pixel phones. And that was massively successful. That Sergey Brin wanted to bring all of that technology and build Google Glass. We built Google Glass, which was not very successful from a product perspective.

Too early, maybe too early.

It was way too early, and as I continue telling the story, you'll see that it happened again in the future. After the Google Glass experience, I decided that working on hardware is not a good idea and continued working in computer vision, object recognition, and things like that as part of the visual search in Google Brain. As part of that, we developed the most advanced neural network at a time called the Inception, and that powered a large number of products within Google, and in particular visual search in Google Photos.

After that, I got lured into working on hardware, impossible stuff again, and joined Magic Leap to build the most advanced spatial computer in the world at the time. And I'm very proud of the impact we've had there. But toward the end of my time there, I realized that the hardware yet again is not ready. The displays are not what they need to be, and our ambition of having a device that would essentially replace a phone and a laptop is is.

The ambition is too big, and the hardware is not filling in the right requirements. But the other thing I realized is that the algorithms necessary to facilitate this experience. If you remember the movie her, where the main character interacted with the operating system, the requirements to facilitate this all day, everyday wearable device required that there is this companion that understands everything that you see and everything that you do, how you feel, how you react, and allows you to navigate the physical world.

So I decided that while we wait for the hardware to emerge and be appropriate for the experience, it is possible to build that interactive experience with a companion. So I started a company called Headroom, where the goal was to tap into video conferencing experiences that everybody was having, similar to the one that we are doing right now, where the AI understands everything that's being discussed and understands you as a participant from a multimodal perspective. It understands what you say, it understands how you say it, it knows when you smile, when you frown, when you're paying attention, when you're angry, and so on.

And then once these models are advanced enough and ready to be deployed, presumably the hardware for the all-day, everyday wearable devices will be ready. And then you can just take the models from laptops where you have a webcam and a screen and translate them into these wearable uh devices of the future. And then we started looking at a bunch of different collaborative platforms for work. Those included a company called Lucid Spark and Lucid Chart and a bunch of others, and Upwork emerged as a very interesting use case because clients and freelancers communicate on the platform all the time.

And being able to capture the essence of their interactions was a critical component in order to be able to improve the quality of the matching and the experience from both freelancers, um, and collaborators. And when we started discussing a partnership, because Headroom developed an SDK that you could plug it in into any collaborative platform, we thought there was an interesting strategic relationship to be had, and that's how Upwork was interested in acquiring Headroom. And since December of last year, I've been at Upwork together with my team in the effort to transform it to be this outcome-driven business.

So that's a very different than than the hardware and personal interactions you've been talking about before then.

Well, I do want to talk about hardware for a second. You said hardware three times, and I'm interested in your views on the state of play and the future. Because in many ways, as as you already mentioned, and when you look today at the ability of AI in terms of its cognitive ability, both in terms of understanding multimodal and in terms of responding in multimodal, it far exceeds its hardware counterpart. In many ways, it is able to do all this because of all the, uh, technological innovations that have happened in hardware.

But we're talking different kind of hardware, we're talking computer chips and that kind of hardware. But like in actual mechanical physical hardware, what do you see as the biggest blockers and and how do we unblock them?

I assume you're you're referring to wearable devices in terms of hardware components.

Wearable devices, all kinds of bots and physical devices that can do things, not just necessarily human, you know, augmented devices, but things that can even work autonomously. It feels like the physical limitation far surpasses the the mental and intelligent limitations.

It does to an extent. If you think of the advances we've made in bits, sort of AI for cognition and recognition and translation and things like that, we've made great strides, but we still don't have models that can think and reason, right? These are still pattern matching machines that answer questions formulated by humans. Machines still can't ask questions, right? In terms of hardware, atoms are difficult to manipulate. In wearable devices, the displays are just not there where we need to.

If we talk about augmented reality with see-through displays, the efficiency of of the projectors that we have are just too low, so the batteries don't last long enough. When you go, like, if you look at your cell phone on a sunny day, you can't really see anything on the screen, so you have to, like, cover it with your hand to create a shadow, right? Where you have this wearable device, you can't put a shadow on the whole park that you're in, so that doesn't work.

Now, with robots, one great example of successful, from my perspective, robots are those that are in an extremely confined environments, like surgical robotics, right? Where you have these like sterile rooms where the patient is always positioned in the same way, or like a packaging robot at a warehouse. It does the repetitive thing many, many times, and it works great. The robots that work in more open environments are things like Waymo cars. And I remember when Stanford won the DARPA grand challenge in 2005, where a self-driving car drove, I think, from LA.

to Las Vegas. We're like, cool, this thing is done.

It's funny you say that, by the way. I just literally this morning got cut off by one coming back from my morning coffee. There was actually two of them, the two of them were were. This is a digression, but like, the two of them were more conscientious of not, like, not hitting each other than anybody else.

That that that's true, because they're very well aware of one another, and they know what the other will do. They've made great progress, and they're like a thing now. I actually let my kids take them. I'm a little apprehensive of my kids getting into Uber on their own, but I'm totally fine with them taking Waymos around them. So that's a great example. But the problem again is that we build these large language models, we build detection, recognition, classification models, but these are all predictive systems, which don't have the model of the world.

And there's a lot of discussion about this in the academic circles. Some believe that just statistically, by observing a tremendous amount of the world around us, either from video or from text or from audio, that's enough to infer the model of the world. I hold a slightly different opinion, where all of the laws of nature that we've figured out, we have to ingest those as priors. So there's some not that I'm a believer in symbolic AI, but the fact that we understand the laws of gravity,

Yeah, there's some basic physics, we understand those they have fundamentally.

to be exactly, they have to be inserted as initial conditions into these LLMs. So if you ask a large language model what happens if you drop an apple from the roof of your house, of course, it'll tell you it'll fall, but it doesn't do it because it understands what gravity is, it does it because it read so many texts online that keep saying that if you drop it, it'll fall, right? So with robotics, it's kind of the same thing.

If things happen according to the fairly simple model of the environment that you've constructed, then it works perfect, right? It doesn't get tired, it hands don't shake, like it just does the same thing over and over again with this incredible precision. But if things get a little weird, then it starts to get confused in the most unpredictable way. Because, unlike us humans, these things don't degrade gracefully, they can just explode, right?

I like what you're saying here because you're making me rethink my own question. The premise of which was physical technology hasn't advanced as much as the software component of these AI and generative AI. But you're basically saying just the same way as there is a hallucination problem with the software, there is a physical hallucination problem with the hardware. And so let let's not overthink how real capable the software actually is.

It's absolutely right. Like, if I ask you, would you trust ChatGPT to answer all of your emails for a day? You would not agree to this, right?

Yeah, it's only for the 5% that I won't trust it with, right? For 95%, Maybe just I would agree. one person, maybe like Yeah, one email, someone exactly.

says, Do you want a bonus? and you're, and the thing says, no, and you're like, all right, problem, problem. So, um,

I've never gotten an email like that, by the way.

There's a it's a training data problem, right?

exactly, yeah. These things go hand in hand. From a sort of material science and physics perspective, the human body, through evolution, uh, can do tremendous things that we haven't been able to replicate entirely with hardware. Very well, it's still sort of on the horizon, right? And the problem is that we always underestimate how hard the problem is. At Magic Leap, which was by far the well since XAI, I guess, the most funded startup without a product that it ever sold.

I think in total, while I was there, we raised around $3 billion, and people thought this was like an astronomical amount, and we thought that with $3 billion, we can solve this mixed reality problem. Cumulatively to date, between Magic Leap and Microsoft and Meta and Apple, I think we're getting close to $100 billion, and we haven't gotten much further than the $3 billion for Magic Leap. So the problem is extremely difficult from a hardware perspective, and while we're sort of chipping away at it fairly slowly from my perspective, we are still far away to having something.

I remember one of the projects at Google X that we wanted to do, we wanted to create a programmable contact lens that would get power from the heat of your eye, and then it would project all kinds of interesting things for you, like in in sci-fi movies. After, I think, five years of trying, we were able to turn on one pixel in in the contact lens, and then the whole thing was abandoned because it was deemed too difficult.

I think that's where you were going in general, when you're talking about cars, was just how hard all this stuff is to extrapolate. It really is, yeah,

What does this mean, you know, in your view, for letting AI out of the lab, out of these confined domain-specific areas like the operating room in a surgical setting or a very specific task? What does that look like for you in ten years?

giving estimates of long horizon is very difficult. I'm not as good as Ray Kurzweil about this, but in a short amount of time, I believe that generative AI will replace the way humans interact with data. If you think about ChatGPT, it hasn't invented any new information, right? So if you're super savvy with Google and other technologies, you can figure out how to write a birthday greeting in a Shakespearean sonnet style, right? It'll take you a while, and not everybody can do it, but what ChatGPT allows you to do is to interact with the same data in a very, very intuitive format.

So it becomes an interface between the human and the information. So I believe in the future, search engines of this form or any kind of retrieval systems will go away, and everything will be a dialogue between a human and a machine in any mode. If you have the wearable glasses, it'll just see what you're looking at, and you can just say, what's that thing on the sidewalk? And it'll tell you what it is, and you can talk about it.

If you receive a document that needs to get a response, you just tell AI, like, write this up and like, reply Well, we're doing to it.

that now, though, right?

We're doing this now, but we're doing this in a very sort of disjointed way. The flows are extremely fragmented. In the future, you'll just have like one window on your screen, your AI companion, and every person is gonna have their own. It's all gonna be hyper personalized. It'll understand everything that you do to it, like, ChatGPT doesn't know anything about you right now, right?

So I'd like to transition us now to a short segment which we call five questions. I'm just gonna ask you five questions, just rapid fire. Tell us the first thing that comes to your mind, preferably short answers, okay? What do you see as the biggest opportunity for AI right now? biology, What do you think is the biggest misconception about AI?

that it can take over the world.

What was the first career you wanted? What did you want to be when you grew up? A physicist, What kind of physicist? Theoretical?

theoretic theoretical physics. My dad is a very famous physicist. I when I was a kid, I wanted to be like him, I guess.

When is there too much AI? Never. I like that. Uh, what is the one thing you wish AI could do right now that it can't?

Be honest.

Very good, very good.

Andrew, it's been fascinating. I think we could talk for hours here. I'm hungry for more.

Uh, yeah, I'm having a great time, too.

All right, well, maybe we'll have to drag you back here. But thanks for joining us and thanks for taking the time today.

My pleasure, thank you guys.

Sam, I thought that was a really interesting conversation. Obviously, Andrew's been around the block a few times and has been behind some of the seminal models that are open source around, you know, computer vision and reinforcement learning and all that. Uh, w- what struck you in this conversation?

I mean, he, he has been behind a lot of these models, and I thought his breadth of experience was particularly interesting. Because, you know, he has built these models, we can all use these models. I mean, literally any listener right now, if you want to go out there, you can download his inception model. I mean, not just his other people, a lot of people worked on it, but download this inception model off of PyTorch Vision and use it today.

And I think that's a really interesting point that didn't come out quite as much as we were hoping would in the episode, that these tools are becoming accessible to people, people can go out and use them. And we talk about democratization. I'm not sure about democratization is the right word, because that implies everyone is gonna have access. I think a better word is meritocracy, maybe, and that sounds a little sort of like, do people deserve it?

But it's not gonna be that everybody can download them, it's only gonna be the people who are willing to learn about them, take the time to invest in these models. I think that's really important. Just having this stuff available isn't enough. That makes it available, but it doesn't mean that you can use it.

But it also means that more will be expected of people, right? Like, what used to be an edge, which is I work hard and I know a lot, right? I've memorized a lot of facts, I've read a lot of books, and I work hard. That is becoming still valuable, right? But knowledge alone, maybe that's the the reason you use the word democratization, right? Access to knowledge and information alone, which used to be a significant differentiator among individuals and among teams, is slowly eroding as a differentiator.

I guess we have to figure out what is going to be differentiating. And one of the things that he mentioned was the idea of tasks that become much smaller and that, you know, the tools are doing more of them. I'm left wondering what parts of these tasks that are worth investing in. When he was talking about search engines, I immediately was thinking, you know, I'm actually pretty good at googling things. I mean, that's, you know, my Google foo is strong.

And, you know, I wonder if that is become not so important anymore, then, all right, what do I need to be working on next? And I don't think we got to a clear answer on that today. What is it we need to be working on?

Well, I think maybe part of the answer is itself figuring out the question, which is instead of what are the tasks that we need to be better at, it's what are the problems and what are the outcomes that now we're capable of driving, right? Because implicitly we pick that which we think can be done. I mean, at the end of the day, right? We're not gonna pick things that are impossible or difficult, or we can't do it alone, right?

Or we can't do it in a, you know, without hundreds of billions of investment and big teams. I do feel like there's a selection mechanism that's going on, that's like people and organizations and teams pick things that are think believe are feasible within that time window. And I think what is possible is changing, however, to your point earlier around willingness to know and learn so that you know what can be done differently. I think there is a huge gap there.

I mean, I think there's a huge gap in knowledge amongst practitioners and managers and team members about how the art of the possible is changing. And so maybe the real question here is, you know, how big should should we dream, rather than what tasks should we be doing?

Well, Shervin, you're right on talking about the uncertainty in our world and the talent uncertainty that's happening. Maybe we should write a report about that.

Maybe we should do that. Thanks for listening. Sam and I are, in fact, working with our teams on a research report about how artificial intelligence and organizational learning capabilities prepare companies for uncertainty. Look for it later this fall, and next time, Sam and I will meet Jeremy Kahn, AI editor at Fortune. We look forward to speaking with you then.

Thanks for listening to me, myself and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MITSMR and BCG. You can access it by visiting MITSMR dot com forward slash AI for leaders.

We'll put that link in the show notes, and we hope to see you there.