Person
1 episodes · 23 quoted claims
Iavor Bojinov says traditional IT projects are deterministic, giving the same answer each time they are run, unlike AI.
“with an IT project, you know pretty much the end state, and you know that, you know, if you run it once, twice, it will always give you the same answer.”
Iavor Bojinov says ChatGPT can give different answers to two users who write the same prompt.
“So think something like ChatGPT, me and you can write the exact same prompt, and it would actually give us two different answers.”
Iavor Bojinov built an AI data analysis product at LinkedIn that in testing cut analysis time from weeks to about a day or two.
“when we tested it, it cut down analysis time that used to take weeks into maybe a day or two days.”
A week or two after the launch event for Bojinov's LinkedIn AI product, no one was using it, which he attributes to lack of trust.
“And a week or two after it, like, no one was using it.”
Iavor Bojinov says the failed adoption of his LinkedIn AI product helped motivate him to leave LinkedIn and join Harvard Business School.
“one of the things that motivated me to leave LinkedIn and join, uh, HBS was the fact that I'd build this”
Bojinov says data science teams typically pick which AI projects to work on, and may not understand the business strategy.
“typically, when it comes to AI projects, it's the data science team that's actually picking what to work on.”
Bojinov says AI projects require early consideration of privacy, fairness and transparency, which traditional IT projects do not.
“So there's this whole area of responsible AI and ethical AI, which, again, you don't really have with IT projects, right?”
Bojinov uses the Netflix ranking algorithm as an example of an external-facing AI project deployed to customers.
“You have external facing projects where the AI is going to be deployed to your customers. So think like the Netflix ranking algorithm, right?”
Bojinov defines internal-facing AI projects as those deployed to a company's employees, such as sales prioritization tools or support chatbots.
“Internal facing project, on the other hand, are deployed to the employees.”
Bojinov says internal-facing projects usually fall to data science teams, while external-facing ones usually fall to AI or machine learning teams.
“internal facing projects are called sort of data science, and they fall under the purview of a data science team, whereas external facing projects tend to fall under the purview of an AI or a machine learning team.”
Bojinov says a study he did with LinkedIn showed experimentation can improve the final product by about 20% on key business indicators.
“in one study that I did with LinkedIn, we basically showed that when you leverage experimentation, you can improve your final product by about 20% when it comes to key business indicators.”
Bojinov recommends letting users opt in as alpha or beta testers so companies can experiment safely without risking brand image.
“a lot of companies have ways of you signing up to be like a alpha tester or a beta tester, and then you sort of get the latest versions.”
Etsy staff came up with the idea of building an infinite scroll feature for its marketplace.
“folks at Etsy had this idea that maybe they should build this infinite scroll feature.”
Bojinov says Etsy's infinite scroll took roughly six months to build and showed no effect in the experiment.
“It cost them, you know, let's say six months to build this.”
One hypothesis behind Etsy's infinite scroll was that showing 50 products instead of 20 per page would increase purchases.
“maybe instead of showing you 20, I showed you 50, then you might be more likely to buy things.”
Etsy's follow-up experiments found that adding a small delay to page loads had little effect, which Bojinov attributes to its unique products from independent sellers.
“adding a small delay doesn't make a huge deal to Etsy, because Etsy is a bunch of independent producers of unique products.”
Bojinov says most products have a neutral or negative impact on the metrics they were designed to improve.
“most products, uh, have either a neutral or a negative impact on the very same metrics they were designed to improve.”
Bojinov says a recommendation system could raise engagement while moving customers toward lower-value activities.
“That recommendation system could move your customers away from, say, high value activities to low value activities for you, whilst increasing, say, engagement.”
People You May Know is one of LinkedIn's most important algorithms and recommends people users should connect with.
“one of the most important algorithms at LinkedIn is this people you may know algorithm, which basically recommends which people you should connect with.”
LinkedIn measured the success of People You May Know by the share of invitations that were accepted.
“the way you measure the success of this algorithm is by basically counting or looking at the ratio of the number of people that were people invited to connect and what percentage of those actually accepted.”
A study published in Science, co-authored by Bojinov, found LinkedIn's People You May Know algorithm affected the jobs people got a year later.
“in this study that was, uh, published in Science and I have a number of co-authors on it, is that a year down the line, this was actually impacting what jobs people were getting”
Bojinov explains the LinkedIn finding using Mark Granovetter's theory of weak ties, which says arm's-length connections are most useful for finding jobs.
“There's this whole theory, uh, of weak ties that comes from from, uh, th- th- this person called Granovetter”
LinkedIn allowed publication of the audit research and then changed its internal practices to track longer-term effects such as job applications.
“The LinkedIn sort of opened their door, they were transparent about this, they let us publish this research, and then they actually changed their internal practices.”