2013
Marco Argenti became Vice President of Technology at Amazon Web Services in 2013 and ran several AWS businesses.
“Vice President of Technology of Amazon Web Services (AWS) in 2013, where he started and ran several AWS businesses”
Topic
Marco Argenti is the Chief Information Officer of Goldman Sachs, and this page collects what he said on two podcasts, Odd Lots and AI in Financial Services, about how the bank uses generative AI. He covered handling millions of documents a month, the computing power AI needs, and how technology work is changing.
Argenti told Odd Lots that in almost 40 years in IT he has never seen developer headcount decline. He told AI in Financial Services that errors are the worst enemy in document management. He also described joining Amazon Web Services in 2013 as Vice President of Technology, running several businesses there.
21 facts from 2 episodes, in date order. These are what speakers said on the shows, each linked to the moment it was said. They aren't independently verified.
2013
Marco Argenti became Vice President of Technology at Amazon Web Services in 2013 and ran several AWS businesses.
“Vice President of Technology of Amazon Web Services (AWS) in 2013, where he started and ran several AWS businesses”
2024
Marco Argenti had been at Goldman Sachs for about five years, and this was his first role as a chief information officer.
“I've been here for five years, and this is the first time that I do like a CIO job”
2024
Argenti says compute power demand rises quadratically with prompt (context) size, and context windows have grown from thousands to millions of tokens.
“the power scales quadratically with the prompt.”
2024
Argenti cites quantization, reducing numerical precision from 16 to 8 to 4 bits, as one technique to cut AI compute needs.
“quantization, reducing from 16 bits to 8 bits to 4 bit precision”
2024
Argenti says GPUs are the best hardware for training large models, aided by NVIDIA's CUDA software, while inference may favor specialized chips.
“For training, at the moment, there's most likely nothing better than GPUs”
2024
Argenti tells his team not to build software unless it is differentiating, and endorses open source with proper reviews as a hedge against vendor lock-in.
“my direction, my guidance to my team is don't build anything unless you have to.”
2024
Argenti says content production such as pitch books and earnings summaries will be streamlined by generative AI.
“Imagine the creation of pitch books for clients where you take templates, you put a bunch of data, you go out and do research”
Undated
Goldman Sachs Chief Information Officer Marco Argenti says the firm receives and processes millions of documents per month.
“we receive and process millions of documents per month.”
Undated
Marco Argenti says errors are the worst enemy in document management with generative AI.
“when it comes to document management, actually errors are like your worst enemy.”
Undated
A vector database stores concepts as vectors, so similarity between concepts can be measured as distance.
“you're basically putting concepts in the form of vectors.”
Undated
Argenti says retrieval augmented generation raises answer accuracy dramatically.
“we see that the accuracy of the answer by doing this retrieval augmented generation goes up dramatically.”
Undated
Argenti describes attention as the key part of the transformer architecture behind large language models.
“it's all about attention. This kind of the key part of the transformer architecture.”
Undated
Argenti uses the phrase 'return on attention' to describe where people should focus limited time.
“how do you measure your return on attention?”
Undated
Argenti says developers ask AI to pinpoint code areas with possible edge conditions or bugs.
“We have examples of developers really asking the AI, you know, to pinpoint what are the areas of this code”
Undated
Argenti imagines an AI alerting a banker to overnight market events and suggesting which clients to call.
“Imagine a future where a banker could wake up in the morning, something just happened during the night, somewhere, maybe in another market”
Undated
Argenti compares AI-targeted client outreach to the shift from untargeted to targeted advertising.
“it's almost like the shift in relationship between non-targeted and targeted advertising that happened several years ago”
Undated
Argenti's AWS businesses included mobile, serverless computing, Internet of Things, and augmented and virtual reality.
“including mobile, serverless computing, Internet of Things, and augmented and virtual reality.”
Undated
Argenti describes retrieval augmented generation as pointing a model to the relevant document sections so it returns answers with citations.
“you point them to the right sections of the document that actually is more likely to answer your question.”
Undated
Argenti says that in almost 40 years in IT he has never seen developer headcount decline.
“I've been in IT, quote unquote, for like literally almost 40 years and I've never ever seen that go down.”
Undated
At Amazon, Argenti found meetings begin with silent reading of a written document, a practice he has brought to Goldman for 30 to 40% of his meetings.
“the first few meetings are kind of shocking because nobody talks. Everybody starts reading.”
Undated
Argenti described coaxing an AI into saying 'I don't know', which required a prompt establishing it was safe to do so.
“I even decided what to say. It's like, is there a role for humans in a world of AI?”
Episodes