Fashioning the Perfect Fit With AI: Stitch Fix’s Jeff Cooper
Me, Myself, and AI · 16 Apr 2024
Source: https://sloanreview.mit.edu/audio/fashioning-the-perfect-fit-with-ai-stitch-fixs-jeff-cooper
Transcript: https://podcasts.opennash.com/shows/me-myself-and-ai/fashioning-the-perfect-fit-with-ai-stitch-fixs-jeff-cooper-b46b61/ (via OpenNash Podcasts, machine-generated)

How can humans and generative AI work together to ensure we're dressing for success? Find out on today's episode.

I'm Jeff Cooper from Stitch Fix, and you're listening to Me, Myself, and AI.

Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MIT Sloan Management Review and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Hi everyone, today Shervin are talking with Jeff Cooper, senior data science director at Stitch Fix. Jeff, thanks for taking the time to join us.

Thanks so much for inviting me. I'm really excited to be here.

Let's get to some fashion basics here. What is Stitch Fix?

Stitch Fix is an online personal styling service. We serve more than 3 million clients we call them in women's, men's and kids. And what we're trying to do is help people get dressed, uh, offer the most convenient way to find clothes that you love. Find clothes that you might have found yourself, and we can find a great way to put them together with other things. And also find clothes that you might not have chosen for yourself and help you push the boundaries of your style.

So we have this kind of unique art and science approach where when you sign up, you're matched with a stylist, one of our thousands of style experts. And then our stylists work together with our data science team and tools that we provide them to help find clothes for you. We send them to you, uh, you keep what you like, you send back anything that you don't want for free. We've been doing this for over a decade now.

We just had our 13th birthday yesterday, actually, and we just passed 100 million fixes sent pretty recently. So at this point, we have spent a lot of time thinking about how to serve our clients the best, how to blend great algorithms and data science with our style experts intuition and understanding and so forth. We really believe in this model as a way to help clients find what they need.

What a great example of human and machine working together. Tell us more about how that actually happens. What does the machine do? what does the human do? how do they work together?

We're really passionate about this model. We've been at it for a long time, really since the beginning. As you can imagine, for any retailer, the idea of sending people things that they haven't specifically chosen for themselves and then they can return them for free feels a little risky, maybe. And in order to do it well, as we think we do, you really have to know your customers incredibly well. So we have, you know, certain things that we think hard about doing.

We have to have great learnings about our customers. We ask them many questions. Our customers are interested in talking to us about their style. They're here to help be styled. And so we think really hard about all the ways that we gather feedback. When you try on an item, whether you keep it or return it, we ask for a lot of feedback. We get a lot of feedback. Eighty five of clients leave feedback on items. We ask for a lot of questions up front.

We ask some questions as you go, and typically clients leave many requests as they go. Shipment.

So you're learning a lot about your customers through these questions and requests.

Then you have the question of, Okay, what do you do with it? A big piece of this is the human approach. Our stylists get to know our clients well. We have tools where our stylists can see the history of all the people that they've worked with, all the feedback that they've given, all of the ratings. And then within those tools, our stylists also get recommendations from our own internal systems about what our systems think might be great for that client.

And so that's really where our machine learning and AI comes in. We think of our tools on the machine learning and data science side as a great way to help our human stylists kind of get in the ballpark. For any given customer, there might be thousands or tens of thousands of items that might, in principle, be appropriate for them. And a person working with a client and trying to serve them in a timely fashion. It's hard to go through every single thing in the inventory and think about what might be the perfect fit.

So a lot of what we do is help our stylists, you know, narrow down with data and algorithms to a set of items that we think are pretty good. That respect the client's requests for both in general their style and also for that particular fix. If they're shopping for a particular occasion, our algorithms can interpret that from the request that they've given and our stylists can see that. And then the last mile, in many senses, is handled by our stylists who know about fashion trends in a way that our algorithms still don't know.

They know about the sort of human emotional connection that they've made with their client, the specifics of the occasion they might be asking for. And so they can really help them figure out, ah, you know, this would be the best thing for you to try on at this time. So we have a lot of tools that we build on the data science side that we try and arm our stylists with in order to help them find the best assortment of things that they can send our clients to.

Lots of data, lots of feedback.

Yeah, 85% of customers leaving feedback seems huge. That just doesn't seem normal for feedback levels from customers. But I guess it makes sense, cause you have this situation where it's in the customer's best interest to let you know as much about their preferences as they can.

That's exactly right. We are a retailer with a unique model and that puts some constraints on us, but also offers us a lot of power and a sort of different kind of relationship that we can have with our clients. We think that that, you know, direct relationship with our clients is the most important feature of our model. And so we do a lot within our product and in our communications to clients to think about how to keep that feedback loop going.

We, as you said, have very, very, very high feedback rates. And and again, these are even for things that people aren't keeping. You know, typically, if you're returning something to, you know, a big box retailer, you're not necessarily gonna live detailed feedback on it if you're sending it back. But for us, our clients know, hey, that helps my stylist out. I'm working with a person here and I'm working with a set of tools. And if I tell them more about what worked or didn't work, then they learn about me faster.

And that also helps our stylists and our tools evolve with our clients. Something that you loved a couple years ago or even a couple seasons ago might not work for you anymore. Or you might feel like, hey, the trends in the place I'm working at have moved on, or I, I've started a new job. I wanna try something new. Getting that feedback as you go is a really great way for our clients to communicate with us and help us keep our understanding of their style really fresh.

Give us a sense of the scale here. You mentioned 3 million customers, how many items?

We're a full size, full spectrum clothing and apparel retailer, both apparel, shoes, accessories. We ship a couple 100,000 fixes a week, and we have a couple of thousand stylists employed. And so there are many moving parts to help this business scale from where it started with our founder originally. Just, you know, putting these fixes together in her apartment. And so that's a big part of what our tools and automation are about are taking a model that is bespoke and human and enabling this kind of connection and empowering the stylists that we have.

So that they can scale that connection to many clients that they might work with and enable our business to scale across the millions of clients that we're hoping to get dressed.

And I have to imagine that generative AI must be pretty high on your radar, too, with all of its, I guess, more cognitively advanced capabilities.

Very much so. We're really excited about all of the new advancements over the last several years. One of the great things about having a great relationship with our customers and a lot of data about our clients is that we can make that data even more valuable with technological advancements. So the data that we've collected, you know, one, two, three, four years ago becomes more and more valuable to us as new models and new kinds of machine learning and AI are developed.

Because we can apply those tools to the data that we've already have and help fine tune those models with that data. Think about how to train new products on the data that we already have. We have been excited about generative AI for some time. We started working on our outfit completion model as a a little example of a native internal to us. What we think of as generative AI process, something that our stylists were teaching a model which items go together in order to help it build outfits in a fully automated way.

So if you go to our site and you've shopped with us, you'll see a popular feature called Complete Your Looks, which helps pick out items that you've kept that we know that you own and liked and pair them with other things that might be interesting to you. Our clients can shop for those themselves, uh, right on the site using a feature called Freestyle, or they can save them for their stylists to notice and talk to their stylists about.

Yeah, I I love the way that this looked and so forth. So that kind of early, hey, we're gonna create some new content using deep learning models based on top of our existing personalization engines was some of our early forays. We also very early on got excited about large language models. We've spent time with them at smaller scale projects, things like crafting dynamic ad copy or helping to improve our product description pages across all of our site.

More recently, we've made better and better use out of their new models. We have a really exciting new feature working with our stylists using generative AI, where our stylists write personalized notes for each of our clients every time they ship them a fix. And we've rolled out a new feature with OpenAI's GPT four that enables a stylist to choose from a sort of starting place, a template. So this is an optional tool where they can get some of the introductory common to many fixes language out of the way.

But also have the model know we've fed it data about what that customer likes and the items that are in that fix and so forth. And so just as a time saver to enable our stylists to write those notes faster, they've got some candidate language that's already set out there. That's a great example of the kind of approach that we love in AI. We're taking something that is our human connection, we're making it faster and easier and more scalable for our stylists.

This has saved close to twenty of note writing time for our stylists, which is a big savings at our scale. And our stylists have been really thrilled with how this kind of a feature has rolled out.

Yeah, it allows them to focus on what really their strengths are, which might not necessarily be note writing, but much more in, like, design and picking the right assortment and all that. Yep,

This seems fundamentally different, Shervin, than many of the guests we talk to. Uh, like, if I'm shopping for a battery, I know what battery I want, I just need to find it. And so I need to communicate to the company what I want, but in this case, I don't know what I want exactly. What's that called? The Polyan-i paradox? In the Polyan-i paradox, we know how to do things that we can't explain. An example I always hear about is shooting pool.

People can shoot pool without knowing anything about trigonometry. In this case, how can we tell the models how to behave if we don't ourselves know what we want or like? It seems like an interesting scenario here where you're in, like you said, a discovery relationship. If you talk to people like me, we don't know what style we want, but I know what I hate when I see it. It seems fundamentally different.

it's more open-ended.

Yeah, much more open-ended and destination-oriented.

Yeah, it's more open-ended, yeah, but I would say I know this was a question to you, Jeff. I would say that, in my view, isn't this a design problem where you don't know what you're designing? W- exactly what shape it should be? It could be automotive, it could be art or something, but there is a lot of parameters and there's boundary conditions, and so you have choices. It's not a risk problem where it's like, yeah, this is a fraudulent, you know, transaction, don't authorize it.

Or like, this is the right offer for this customer. At this moment, I know exactly. Send him this out of these three promotions. This is different, cause it's so open-ended, and maybe there is not just a global, you know, optimal. Maybe there's many. And anyway, I mean, Jeff, you're talking about a ongoing relationship.

very much so.

Like, if I'm in a relationship with Sam, I'm not trying to optimize every single interaction, I'm just trying to, like, have a good relationship.

What you said, Shervin, resonates so much about this being, you know, something where we don't know what the right, the quote unquote right end goal should be. And and it's quite difficult to design, you know, on the data science team, we think a lot about there needs to be an objective function on these models. What does it need to be? And we have a lot of debate on the team about exactly how to model our client happiness and satisfaction in a way that the models can steer in the right direction.

One of the reasons we're so passionate about this combination of humans and ML is that, first of all, it enables us to solve some of those thorny problems. By saying, well, the humans will do some piece of the company's objective function, and the models will do some piece of the company's objective function. And both of them will contribute the things that they're best at. So that we can help make our sort of, you know, our overall client outcomes the best.

A really interesting thing, you know, thinking about design space and and how machine learning models can help with these fundamentally creative problems. Is we see both patterns within our usage, where the models are helping our stylists get in the ballpark, and then our stylists kind of narrow down and find the last mile. But we also see see patterns of the other kind, where our stylists are fundamentally describing some core constraints, and then our models are nailing down exactly where they want to land.

Our outfit model is a good example of this, where we spent a lot of time with our stylists helping them train the model. And a lot of what the stylist training was there to do was to build kind of guardrails into that model. That say, you're never gonna have this kind of pant go with this kind of jacket. These are pajama pants, they cannot go with a nice blouse. Those kinds of fundamental guardrails, both in hard business logic, but also just in repeated training and helping the model understand the kind of core concepts.

And so, you know, we think in many parts of this creative process, there are places for, you know, both the machine to provide the core search space that then the people are working within, and the humans to set out the core space that the models are then working in. And which you use depends a lot on the specifics of the product feature that you're trying to design and the scale. So for something like our outfit model, we're trying to create tens of millions of outfits a day for our clients.

We cannot have human beings put all of those together every time. For our fixes, we must have our stylists really involved in that process because that's one of our core promises. And so we think, depending on the kind of feature, the kind of scale you're working with, there are a spectrum of possible interactions between the human and the AI model that can help the company produce the best outcome.

Yeah, no, that makes a lot of sense. And I mean, when you made the comment of this is a pajama top or bottom, doesn't go with this, I mean, it doesn't go with this now, but it might, right? I mean, you think about Exactly, for sure, and so it seems like it's a ongoing dialectic, or maybe a triadic, right between, like, the I think that's the stylist

exactly right. It is a three and the machine legged

and the customer, right?

very much so. If you want to even make it a little more complex, as we love to do in data, it's really a a four point problem where the fourth is wider fashion trends. Exactly to your point, where we have this ongoing evolution of what our customers are seeing out in the market, what they're seeing out in fashion, out in the world, what our stylists are seeing as up and coming trends. That our customers, you know, might not be aware of, or or might be aware of, but don't think that they're sort of right for.

And our stylists can kind of see something within our clients and say, like, actually, I think you would look great in this, even if you don't think so. And our models, which can help pick up some of those trends in the data among other customers that are similar to that one. So it does end up being this really interesting set of conversations between all of those points.

One of the things that struck me, I, you're right, I was immediately going to what? What is the loss function I'm optimizing on? But, you know, there's, I think, maybe the nerd not fashion in me. But actually, as you're talking about this and you're talking about pajamas, don't fit with this. Like, even I know that, you know, I don't wear dark socks with sandals. But it seems it must have been frustrating for your stylist to have to teach a model all those things that we take for granted.

But you started, you know, with Do do a an initial

you wear light socks? human. Do you just just so so we know? do you wear light socks? Because

So so wait, you're telling me there's no right sock choice for a sandal? I I've my fashion world is is thrown asunder It's all about here.

confidence. If you, if you know what you're trying to go for, Sam, you can wear it.

yeah, you should definitely follow me for fashion trends. But you started off with a very human world, and now you're in a very augmented world. Over the course of these 13 years, it seems like there must have been some frustrating toddler years in there. Where your stylists have to be saying, I can't believe this stupid model put this together like this. How do you work through that?

very much so, and you know, really interesting history, you know, I wasn't here at the very beginning of Stitch Fix, but I've heard about plenty of stories in developing the models. You get started where you can, and the process of getting stylists comfortable with the scores that our models are producing is an ongoing one that we're always still talking about. In the early going, you have basic recommendation models, and even 1012 years ago, people had a good sense of, hey, a simple scoring system that is gonna say these other clients that have bought similar things might also be interested in.

This kind of thing is gonna help you narrow down to, you know, a set of items that might be useful. That's something that any stylist, really, anybody working in retail, can understand. And we've just layered on improvements and complexity since then, working really closely with our stylists, where they request a lot of features and request a lot of things, both changes to the model or additional information that might be helpful to them. One of the spaces that we've been working hard on and considering where it might be useful is as we now have been around for many years and many of our customers have been with us for dozens and dozens of fixes.

You know, for a stylist to come in and look at all of the feedback that they've given over years potentially can be really complicated. Well, with our new generative tools, we have the possibility that we can create summaries of those things and help compress some of that information a little bit further. In this case, almost like you have a a stylist having a partner that can help do some of the extra work right alongside them. But how we think about talking to our stylists about why is this score what it is is a really complicated problem for any human in the loop kind of system.

It's not one that we've solved at all. We do a lot of training with our stylists. A really big advance for us in the last couple years was moving our models to a single unified recommendation model. One of the toddler steps, as you say, that we we took along the way was having, you know, here's a a machine learning model for women only for fixes. Here's a machine learning model for women only for the freestyle portion of the site, the clients shopping on their own.

Here's another different model only for men, and having even several different models that might be used at different points in the fix journey. A big advance that we had was to help bring all of those models together into a single centralized place where we can gather all of the information about all of our clients. And now, at least day to day, client to client, stylists can feel like, okay, this model always knows all of the information about the client.

As opposed to, oh, when they're shopping over here on this part of the site, it doesn't know things that I, as the stylist, know that this customer has bought.

That seems really easy to say, but really hard to do.

Hard to do, and a lot of it comes down to this explainability question. This interaction that we have between stylists, customers and models to take the social portion out of it for a minute. Any machine learning system has to face the question of explainability of the people that are using it or getting outputs from it. Often need to understand something about why these things were generated, and that's a hard enough problem to solve. Just, you know, when you're talking directly to a customer, you know, if I look at my recommendations on another retail site, I might be like, what, why is this being recommended to me?

I don't quite understand. Many different people have tried to solve this problem in different ways. We have the additional complexity of also our stylists need to understand where these recommendations come from, and our stylists need to explain those recommendations to our clients. So we need to be able to find ways for our stylists to have a sense of what is the model's thought process in some sense. And then for them to also be able to explain why these things might have, we think, as a kind of human plus model combo, been a particularly good choice for our clients.

That's something that, you know, as a great example, we think our stylists are still really, really, really expert at. It's quite difficult to beat, even with advanced language models, the power of a person who knows their domain well and can talk through why. For you, an individual, this piece might be the best.

so we talked about the wide, you know, gamut of items and customers and data, and you have, you said, several thousand stylists. How is AI helping them learn from each other? Cause when you were talking about our stylist, I'm thinking it's just not a homogeneous group of people and they have

That's right.

different tastes. And they could learn from each other, or they could challenge each other. So how are you doing that?

We work closely when training them on the latest and greatest for our machine learning models and our tools. What are the things to be aware of this season, or as new merchandise and apparel rolls in for the current new month? And so much of that training is done at the human level to help them understand. Here's the things to look out for, here's the things that are gonna seem new. A great example here was the rollout of this generative AI note writing template, where the training that we're doing for that varied a good bit.

Depending on are you someone that has been writing those for many, many years on your own? Are you someone that has seen other attempts that we've made to do note writing tools, or are you coming into this fresh? Our research really suggests that our clients are looking for more interaction with our stylists as humans, and we think that's the really exciting next frontier for us to be helping our clients to understand. Hey, from the very beginning, we can talk to you a bit about why this stylist has been paired with you.

Any style expert is going to have a sense of the customers that they really resonate with, the fashion trends that they really resonate with. We have all of that information, and that's a really exciting area that we're thinking about finding ways to surface better to our clients.

It's wonderful. Tell us how you ended up, where you are, what was the journey like?

I don't see a lot of cognitive neuroscience in this so far. Is it there?

Um, I'll say, we have a wonderful team of people here, many of whom came from scientific backgrounds, I'm sure, as many other of your guests have talked to you about. Background in academic science ends up being a wonderful set of experiences to learn about how to interact with real data. We have a little bit of a running gag at Stitch Fix with our people with social science backgrounds, like myself, coming from psychology. Other people have come from perhaps economics or other social science backgrounds, talking to our our partners who have physical science backgrounds.

And you get sort of a somewhat different exposure working with data. If you come from astronomy or geology or chemistry, you might have a sense of how you expect data to behave. Stars, they are a little bit different from each other, but Yeah, stars follow rules, yeah, they they follow rules. If you spent your

humans don't.

your academic background cutting your teeth on working with college undergrads or little kids, you know, or even grown ups out in the world, you understand variability in data at a more visceral level than you might otherwise. So I I got into data science in part because I'm interested in people, I'm interested in human behavior. I just think people are the most interesting, complicated things in the world. That's why I got into psychology in the first place, and data science ends up being the field where there is the most data about what people do.

And so a lot of what I think about day to day still really resembles thinking about our core theories about decision making that I was doing back in grad school. You were saying earlier, Sam, it can be hard to figure out what the objective function is for fashion or for an outfit to put together. If you spent time in psychology, that's all you I'm just used to think about

saying everything.

those kinds of problems. Everything you would do is trying to take this messy, amorphous human concept and turn it into some kind of mathematical model just to be able to measure it and quantify it. And so I think you have to get really comfortable in data science when you're working with real customers, especially in businesses where you are working directly with customers. Who are not going to do exactly what you think they're going to do at any point, and are not looking for exactly what you think they're looking for at any point.

You have to get comfortable with the idea that you need to take something very squishy, something difficult to render into numbers and find a way to turn it into numbers so that you can measure it. I think data science is a wonderful field for the art of using math, using statistical modeling, using high precision computing tools to actually say something interesting about these really complicated, hard to predict things about how we feel and decide, and how we feel like we look good, our self-image.

Uh, that are. They seem very difficult to put into numbers, but when you put them into numbers, you can often learn a lot.

That's a great answer, and it opens, you know, our minds, that data science doesn't have to be only in fields where an exact it's not exactness necessarily. In fact, the amorphousness that you're talking about and the wide range, the spectrum of possible very good solutions to something is a great tool for open-ended problems, which are actually a lot of you all of human problems are open-ended that way.

statistics and machine learning are both fields that are fundamentally about dealing with variability in data. They are about dealing with problems where you do the same thing and something different happens, and nothing is better for variability in data than fashion. Two people look at the same thing, and one of them thinks that is amazing, and one of them thinks, eh, not for me. And being able to, you know, take that very human variability and turn it into something that you can approach with numbers, try and make some predictions about, try and summarize at scale, I just think is a incredibly fascinating problem.

Shervin, are you five? Questioning? I usually send that to Shervin, like on the Slob, and I'm not tech.

The five questions, let me pull that up. So, Jeff, do you know about the five questions?

I don't think I do.

That's wonderful. That's supposed to be the Sure, it likes one. So,

it when people get blindsided.

yeah, we have a short segment where we ask you five random questions.

I like that you've described this as a verb. Are you five? Questioning

So we're gonna ask you five questions. Tell us the first thing that comes to your mind, just sort of rapid fire type. What do you see as the biggest opportunities for AI right now?

how to get it to work with humans?

Wonderful. What is the biggest misconception about AI? What do people get wrong?

That it's smarter than humans?

What was the first career you wanted? What did you wanna be when you grew up?

An astronaut?

When is there too much AI?

When it doesn't leave space for people

What is the one thing you wish AI could do right now that it can't?

operate in the physical world more, we're very excited about, and when I say we here, I mean, I think, you know, in the data science and and our AI community. About all that we can do with information and language, fantastic, super important. There's so many more opportunities to help people. When we think about not just robots, but sort of automation and physical automation, um, starting to be more linked to these more cognitively powerful models that we can interact with in a more human way.

A and and really, we think, you know, a lot of what has been so interesting to everybody about the large language model moment and the and the big advances is that it offers the opportunity to interact with these automated systems. Much more like you would interact with a person, that's what people wanna do. And so unlocking the ability for us to interact with automated systems that can operate in the physical world more in a more human way, I I think, is an area I'm really excited about.

That's wonderful.

Uh, that makes a lot of sense. Jeff, we appreciate the time you spent with us today. It's been so fun, It's it's really fascinating how you're using AI to help your stylist and customers learn more about themselves. In this case, everything you've mentioned has been about learning, bidirectional learning, even. I think I hadn't appreciated that symbiotic relationship before today. Your models are exploring a space, and your stylists are helping the models explore the space. That feedback in loop seems really important.

I also hadn't appreciated the complexity. These things always sound so simple. Oh, yeah, use some AI to help solve this problem, but the devil is in the details. I gu- I guess in this case, the devil doesn't wear Prada, the devil wears silicon.

That was very good. Yeah,

And I like the phrase about getting started where you can. I think that's such a good phrase, Shervin, that's one we can pick up on. Get started where you can. Thanks so much for talking with us today, Jeff.

it's been so great. Thanks for inviting me.

the devil wears silicon. That was really good.

thank you, thank you. I Did you just come actually just up with that I just right now, or did that you were

you work on it?

like, Yeah, I mean, I do. I wish that's

Very good.

Thanks for joining us today. Next time, Shervin and I talk with an AI startup founder who starred in a recent Spielberg film. Get ready, player one, grab your popcorn and tune in in two weeks.

Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation. From MIT Sloan Management Review and BCG, you can access it by visiting MIT Sloan Management Review dot com forward slash AI for leaders.

We'll put that link in the show notes, and we hope to see you there.