SAP's Business Technology Platform makes more than thirty large language models available to developers.
“we actually make more than thirty different large language models available.”
Person
Walter Sun
1 episodes · 22 quoted claims
SAP loaded its help and consulting documents into a retrieval augmented generation system so consultants can ask questions in natural language.
“we effectively uploaded with retrieval augmented generation our help documents, our consulting docs.”
SAP's annual Sapphire conference was scheduled for May 19 to 21 in the week after the recording.
“we're having our annual conference, Sapphire Conference, May nineteenth to twenty first next week.”
Walter Sun says SAP is not building its own large language model and instead builds on existing ones.
“we're not building our own LLM because that area has been well researched and well developed.”
SAP held internal AI Days in 2023, with sessions across its global units to teach employees about large language models.
“in 2023, we had what we called AI days, which we delivered a bunch of sessions across different global units”
Walter Sun says free public chatbots may use users' questions to improve their models, which creates confidentiality risks for businesses.
“Because the trade off is you're actually allowing them to improve their models on your questions, which is fair for them, but it's not good for a business”
SAP shipped a feature with its SuccessFactors team that uses large language models to generate job descriptions from short natural-language prompts.
“job description creation with a feature we shipped over a year ago, where you use large language models and you just use natural language.”
Walter Sun uses a best-fit line with two parameters as an analogy to explain model parameters.
“best fit line, if you remember, is y is equal to ax plus b. Your two parameters are a and b.”
Walter Sun says language models with around 100 billion parameters are now considered large.
“now we're looking at like 100 billion parameters, so that's considered large.”
Walter Sun states that there are roughly 300 commercial airports in the world, used to illustrate a tuned travel agent.
“There's actually 300 airports in the world that are commercial, and people typically fly from these.”
Walter Sun argues small agents can reach about 95 percent of a full language model's performance in their area of expertise at lower computing cost.
“you can build a smaller agent. So it's a smaller computational cost. And then they can still perform, at, let's say, ninety five or close to the full language models”
Walter Sun describes test-time compute as letting a model take time to reason before answering, so it can iterate or consult other agents.
“Basically, that's a fancy technical term of, like, you ask a language model a question, instead of saying, give me the answer right away.”
Walter Sun describes a company-specific travel agent that learns through reinforcement learning to deprioritize red-eye flights if employees avoid them.
“So if that's the case, then I'll lower the ranking of red eye flights and prefer just daytime flights.”
Walter Sun says vanilla language models are weak at math, though newer reasoning agents can solve Olympiad-style problems.
“language models, in fact, aren't good at math. You know that there's all these stories about the fails they had”
SAP is building a tabular foundation model for business data and predictions to complement large language models.
“we're actually building a tabular foundation model to complement the large language model.”
SAP's tabular model uses the same transformer technology as large language models but is trained for business data and predictions.
“Using the same transformer based technologies that are used by LLMs for this tabular model, and this model is focused on business data and predictions.”
Walter Sun says SAP can find industry-wide patterns, such as the Black Friday to Christmas retail surge, from anonymized data of customers who opt in.
“it's the retail business in North America. And so if every company is opting in and we've anonymized the data”
Walter Sun worked at BlackRock on quantitative fixed income work before returning to complete his PhD.
“I worked at BlackRock doing quant work on fixed income.”
Walter Sun worked at Apple on early versions of QuickTime.
“working on some of the early versions of QuickTime, which is now integrated into, like, you know, iTunes and other products.”
Walter Sun started Bing Predicts at Microsoft, a product that used big data for predictive analytics.
“I actually started a new product called Bing Predicts, which was to use big data to do predictive analytics”
Walter Sun says Bing Predicts used aggregated, anonymized web search traffic to predict political affiliations.
“We're actually predicting political affiliations based off of aggregated and anonymized web search traffic.”
Walter Sun joined SAP from Microsoft to build generative AI for its customers and products.
“I joined SAP a couple years ago and have been working on building out generative AI for our customers and our products.”