Leading a Workforce Empowered by New AI Tools
HBR IdeaCast · 29 Aug 2023
Source: https://hbr.org/podcast/2023/08/leading-a-workforce-empowered-by-new-ai-tools
Transcript: https://podcasts.opennash.com/shows/hbr-ideacast/leading-a-workforce-empowered-by-new-ai-tools-53edb7/ (via OpenNash Podcasts, machine-generated)

Welcome to the HBR IdeaCast from Harvard Business Review. I'm Alison Beard. When any new technology comes into a workplace, you'll usually see the IT department and a few other early adopters experimenting with it first. Eventually, though, as the tech becomes more user-friendly, there's a tipping point where almost everyone could find a way to use it in their jobs, from the C-suite to the back office. Right now, artificial intelligence, and specifically generative AI, is having that kind of moment.

Once the purview of mathematicians, engineers and coders, AI tools can now be effectively designed and deployed by anyone who has a basic knowledge and training. But how do we as individuals figure out how to do it most effectively? How do managers both encourage and corral these efforts, and what strategies should organizations put in place to harness the power of what our guest today calls citizen development? Tom Davenport is a professor of information technology and management at Babson College, a visiting scholar at the MIT Initiative on the Digital Economy, and co-author of the HBR article We're All Programmers Now, as well as the book All In on AI, How Smart Companies Win Big with Artificial Intelligence.

Tom, welcome.

Hi, Alison, happy to be here with you.

Could you start just by illustrating really clearly this idea that everyone, me included, is or should be a programmer, a citizen developer now? How do you see that happening on the ground at organizations with people in non-tech roles?

Well, you know, this is a trend that has been developing for a while. I mean, I think even my traditional area of analytics and big data and AI. We've seen these tools getting easier and easier to use, and it's just point and click now for many interfaces and so on. So you didn't really need to have a lot of technical background. And then a couple, maybe five years ago, you started to see these quote low-code, no-code tools emerging that would let you build small systems without really having a lot of technical background or needing to produce programming code.

And then there were automation versions of that, the robotic process automation vendors introduced tools that were very easy to use, and people could create their own kind of workflow automations. In citizen data science, we've seen, uh, great advances with automated machine learning tools that that are quite accessible. And now generative AI really takes it to the ultimate level, where if you can write an English sentence or whatever language about what you want, it can produce code or do a data analysis or or do almost anything technical that you would like.

And so I think the the last barriers have been erased to non-technical individuals being able to do almost anything that they would like to do with information technology.

So give me some specific examples of people who I wouldn't think of as being technologically focused in their roles, putting these tools to work.

Well, we're one of my co-authors on the HBR article. His name is Ian Barkin, and I are, um, working on a book on this topic. There are a number of people we've found in large corporations who are starting to do this sort of work and having a big impact. So I haven't gotten permission to use his name, but there was a gentleman who was employed at Home Depot, who, um, had been doing a lot of spreadsheet work.

He was in the area of forecasting and assessing consumer demand and looking at what that means for, um, what kind of inventory they should have around the different Home Depot stores. And really a very time consuming job, basically, um, lived on spreadsheets. And he discovered a tool, in this case, it's from a vendor called Alteryx that lets you automate a lot of the data inputs and outputs and some of the analysis as well. And so, you know, he was able to produce a sales forecast in a few hours at most, rather than the weeks that it took him previously.

And Home Depot made a huge amount of money. They cited, um, uh, well into the nine figures of extra income as a result. And, um, he got employee of the year. Now, as often happens, he's no longer doing that. He's now a consultant to help other people use these tools. So I suppose he's moved from being an amateur to a professional, but that's quite common. We've seen it at a number of other companies. Um, talking to somebody in Lego recently who's done a similar kind of thing.

I, I was talking recently to a fellow at BMW who I wasn't talking to him about citizen development, but turns out he heads that area as well. And he said, Well, we're we're training 80,000 people in citizen development approaches, so it's really, um, going viral.

yeah. My worry when you were telling that story about the fellow at Home Depot is that he was putting himself out of a job. Now, obviously, he moved on to a much better one. But is that a danger associated with citizen development?

Well, you know, there seems to be enough development around that anybody can do it, and there's still plenty more to do. You know, uh, almost every company these days wants to digitize, and, you know, we have historically relied on IT departments to do that kind of work. But I think almost everybody has a story of asking for an application from IT and being told, Um, sorry, you know, we don't have time for that. Or, sure, we'll, we'll get to it within a year or two, and nobody wants to wait that long.

So I think this just radically accelerates the timeframe in which we can get the technology capabilities that we that we need, and there there seems to be no limit to them.

So is it your sense that sort of the average non-tech employee is eager to jump in and learn how to do this, or do you think that most people need some incentivizing?

I think it depends. I mean, if they are sort of technical heat seekers, then they're probably anxious to do it. They also, I think, um, typically aren't quite sure what this will mean for their career path in many cases. And, you know, you've typically either been a professional IT person or you've been a non-IT business person, and never the twain shall meet. But now I think there are more and more hybrid kinds of roles possible. In many cases, I think you do need a little bit of instruction in how to do this well.

But I think, uh, there's a large population that is at least open to it with some instruction.

On the organizational level, do you see interest in and applicability of citizen AI development differ across companies, industries, geography? You know, does it have to be a company that's already focused on using AI or not?

I think it's bubbling up pretty much all over the place. As I mentioned, you know, consumer products companies like Lego, PepsiCo is another one that I've talked to, not historically known for their AI activity. So I think it is bubbling up in places that have not historically been heavily AI oriented. And maybe one could argue that the places where it's least common, the industries where it's least common are those that are very sort of transactional in nature with their IT systems and where there's a fair amount of regulation.

So you don't see as much of it in banking, for example. One of my co-authors on the HBR article was head of data science at, uh, a big European bank, and he's since, um, become a consultant himself. But he said, you know, there was some opposition, and, you know, there probably should be. You don't want your basic demand deposit accounting system that keeps track of how much money you have in the bank to be generated all over the company in different ways.

But, um, I think there are lots of possibilities in almost every industry. I did recently talk to a very prominent technology company, and they've recently introduced a number of tools for doing this kind of work, both on the data analysis side and on the automation side, and also quite big into generative AI. And I think they're right on the edge of chaos, they're not over the edge yet. But, you know, thousands and thousands of different dashboards being created of small automations, small departmental-level programs, and they're doing their best to try to maintain some control over all of this, but it is certainly challenging.

yeah. So let's get into some of those challenges or pain points. What are the big ones that arise when people start either doing this on their own or with encouragement from their managers?

Well, I think, you know, most companies are not that far along yet. But certainly you worry about someone developing an important application on which the organization comes to rely. And, you know, maybe it's not well built, maybe, um, some of its calculations are not accurate. Um, maybe that person leaves and hasn't documented the system very well. I mean, one of the good things about generative AI is it's also quite good at documenting systems, not just at, um, creating code.

So, you know, maybe we'll see some help there. But if we become highly reliant on systems that are really quite, you know, critical to the success of the business, that I think requires some controls. It requires, you know, sort of registration, uh, uh, uh, by some central group, maybe IT, uh, maybe a review of the functionality of the program to make sure it's up to standard and and so on. Um, and some companies have done that, and we talk in the article about PwC's efforts at citizen development.

And they have a a group of people who were amateurs for the most part, technically, who, um, have been charged with taking them into the digital age and digital accelerators, they call them. And they're encouraged to submit all of the applications they develop to some central, uh, hub, and in that situation, they're evaluated. And if they work well and they appear to be of value to people around PwC, beyond, you know, their own little groups or or selves, they will make them broadly available.

And they even, you know, pay a little bit to the to the person who contributes it, depending on, you know, how much usage it gets. So that's the role

That the IT department plays in this new world, you know, oversight, quality control, that type of thing.

That is certainly one important role, and I think broad-minded IT departments can play some other roles as well. They can provide a lot of the necessary training if that is deemed appropriate. And really, there's a kind of a community development aspect of this, where you encourage not only encourage people to use these kinds of tools, but also to share what they've learned. And to get together in regular meetings and hear updates about the technology, or hear what great things other members of the community have done.

So I don't know who else is gonna do that in general, if it's not IT, or, you know, if we're talking an automation or an analytics oriented group, they may not be officially in IT, but they can still sponsor some of those community development kinds of activities.

Yeah, and I imagine for those groups who have, over the past decade, you know, been flooded with demand from colleagues to help them, you know, do all of this work, this this will unburden them in some way if they can figure out how to do it right.

There is that appeal. On the other hand, I think many IT organizations have been resistant to this for quite a while. They don't think amateurs will cre- be able to create high quality code. They are worried about, uh, uh, a citizen-developed application being dumped on them and say, fix this up, um, it has a problem or update it or something like that. They'd rather develop their own code, so I think only slowly, and in some cases with reluctance, is the IT organization moving toward this view of the world.

In some cases, um, we've talked to a couple of companies where they got a new chief information officer and he or she realized, you know, this is the way of the future, the way things are going. And so they started to open the open the floodgates a little bit. But after, you know, many years of resistance,

What role should team leaders be playing, you know, to ensure that all of these efforts are happening effectively?

well, I think that's probably the area where you're most likely to see the, you know, the motivation coming from. If, you know, that Home Depot forecaster hadn't had a boss who was tolerant of that sort of technical innovation, then it probably wouldn't have been successful. At Lego, we talked to someone who's a demand forecester, and I hadn't really thought about this previously, but it appears, you know, demand forecasting involves typically involves a lot of different types of information that you're trying to pull together from across multiple systems, internal and external data and so on.

And so it can be very labor-intensive to pull all that information together unless you automate the process. And so it worked quite well at Lego as well to do that. Initially, the IT department was not supportive, but the head of demand, uh, planning was supportive of it. And that, you know, kind of team leadership role, I think, is arguably the most important one in in making this happen. Both in terms of providing the encouragement and also ensuring that people are following the necessary, you know, control processes.

And at the higher level, you know, maybe C-suite leaders, for example, how do they think about maintaining control over all of this AI development, if control is the right word?

You know, I think there's so if we're talking about generative AI, there's so many different things that an organization can do with it. I was just looking at a survey that I did with the, um, MIT Chief Data Officer Group Symposium, the second annual survey sponsored by Amazon Web Services and data just came in. We have strong focus on generative AI. Um, I think that software engineering was the third most desired use case, or most focused on use case in there.

Um, but in another question, sixteen said they were banning. There was no authorized use of generative AI in their companies. I think in general, that's a bad idea, but you know, it, it's it at least shows that an organization is paying attention to it. I think almost every organization today should have high-level meetings about a Are there some ways that we can use this in our organization? B What sort of risks and concerns do we have about its use?

C What kinds of policies do we need to put in place to make it more effective? And that's whether you know you're talking about generating code or generating marketing blogs, or using it in customer support. Any of those things, I think, require some deliberation about what's the organization's strategy and what controls does it need to put in in place.

And are there any examples you could point to from organizations that are finding a good balance between keeping things open and nimble, but then also protecting against some of the dangers we've talked about?

Funny. Yesterday, I got an email from the chief data and technology officer of a large marketing services company. They identified five or six key areas in which generative AI could be used within the organization. They said, in general, our philosophy is to make this a kind of decentralized activity, so we don't want to apply too heavy a hand, but we we want to sort of see what people are doing. So they looked at a whole variety of use cases and said, What's the status of this particular use case?

Is it in production? And there were very few of those that were, I think, in the AWS survey I was just looking at, only eight of organizations said they had systems like this in production. Is it a proof of concept, um, where we can see whether it works or not, or is it just an idea? So I think kind of inventory of things that are happening around the organization is a very good idea. And they, like many other organizations that I've worked with, have decided that the risk of using a public generative model is too high.

So we need to do a deal with OpenAI or with Google or whatever provider they are using for these models. And say, we don't want our prompts to make it into your model. That solves a lot of the problems relative to intellectual property ownership that a lot of companies seem to be worried about.

Yeah, and you also mentioned training, so what types of training do you recommend, or what are some examples of of good programs you've seen?

Well, again, it kind of depends on what sort of citizen you want to empower. If, for example, it's generative AI, then you need to tell people you know what types of prompts are likely to be effective for producing code and how to interface with the needed data and other transactional systems that you may be working with. So, um, that probably would be run by an IT organization or a, um, an AI organization, or if your company has one, if it's automation, um, typically we find that's being done more by some companies have have s specific automation oriented groups, but a number of them, um, are primarily focused with process improvement.

And, you know, being a former process reengineering person, I'm I'm very interested in the idea that, you know, we can produce code that will automate workflows within organizations. But, um, in those cases, you not need to you not only need to know something about the tools that are used, but also you need to know something about process improvement. But, you know, you can save some real money that way. If it's data science that you're trying to empower the citizenry for, there is some sort of probably statistical training that you're gonna make sure that people already have, or you can provide it.

You know, it's, um, relatively easy to find a lot of that stuff in online courses these days. But, you know, the software is now quite easy to use. And, by the way, there's a generative AI version of that called Code Interpreter. Um, it's a beta offering from OpenAI now that lets you do basically, um, machine learning analysis with a very simple prompt. You're basically saying, um, you know, um, here, use this data set, and here's the the, um, particular variable or feature that I'm trying to predict.

And tell me, you know, which variables in my data are likely to be good predictors, and how, how good a job can they do? I did this the other day, a two-line prompt for a little data set, two-line prompt gave me three pages of data analysis with a machine learning model applied to it. It's just quite astounding.

How are you seeing companies identify the people who will become their citizen developers? Is it, you know, something that they offer this training to their entire workforce, a certain subset of the workforce, or is it mainly a volunteer army?

I think it's, um, mainly volunteers. There are some companies, I mean, J and J has a s a series of criteria that they apply to you once you volunteer. Do you have any experience with data? Are you, um, um, technically adept, et cetera? Some other companies will take anybody who who comes along and figures, even if they drop out, it'll advance the cause, um, of citizen development if they get at least a little bit of training.

There are some organizations that have certifications. I talk, I do a lot of work with Deloitte, and they have certification of citizen developer, citizen data scientist, and so on. You have to go through certain levels of training to to get to each level. You know, I think it depends in part on what you're trying to accomplish. Are you open to a broad democratization of these capabilities, or do you really wanna be a lot more targeted about it?

And and a little bit more careful about who's doing the work?

You have, uh, talked about sort of hours saved, money saved, and you talk in the article about how important it is to track these measures of of value creation. What are some of the effective ways that you see companies do that? When you, when you say that this company has saved x hours of productivity, or this company has saved x amount of dollars, how did they track that? how did they measure it?

Well, yeah, and I, by the way, I think there's a big difference between the minutes and hours saved. I think it was ATT that said, Oh, we've saved 13 million minutes from our automations. And my immediate question is, Well, what are people doing with all that freed up time? Are Right? I they?

think about that Home Depot guy too, like when he when he automated his job, what happened then?

yeah. I think in his case, he moved on to other aspects of production forecasting and got more detailed about, you know, the different product categories they would would use it for. But I think in general, it's not a good idea to just look at hours or minutes saved without a sense of what people are doing instead. And so, uh, I think the smart organizations I was talking to, um, an individual at PepsiCo lately who is in the finance organization and sponsoring a lot of citizen automation efforts in particular.

And he said, every one we try to figure out, what's the economic value of it? Is it costs avoided? Somehow? Is it, you know, a hire that was prevented? As a result? Not too many people seem to get laid off as a result of these things yet, but that could be another source of savings. Is it making, uh, better decision? That's, you know, kind of yielded more, um, sales in a particular domain, uh, so has it increased the bottom line, or, um, uh, the top line?

And or reduced cost? You know, the various categories of economic benefit you can say, and, uh, I think that's the only way you can really justify a lot of these activities. Um, and typically, you know, you want somebody to certify that and not just have the, you know, the IT or automation or data science people do it to have the finance organization be behind it as well and kind of certifying the the outcome.

Yeah, and so if you're the person who's, say, working for a a small or medium-sized company that's, you know, very far away from the tech sector. And and you wanna try this, do you talk to your manager about it? Do you experiment with it first and then show the results? You know, as we talked about earlier, is there a danger there that you then just put yourself out of a job?

Yeah, I mean, I think employers don't own twenty four hours of our time. So if you want to explore, um, these capabilities on your own time, I don't see that there's anything wrong with it. But I wouldn't take it very far without, you know, after you demonstrate to yourself that it's a workable solution for improving your own productivity. I would, um, talk to my boss about it. And and I think I'd say that with generative AI, you should, if you're going to use it, you should inform your teacher, your boss, your whatever the stakeholder is, that you're going to use it.

Or it's probably not going to be good for your career in the in the long run. Um, I don't really know. I haven't done enough code generation with generative AI to know h how obvious it is, um, but I've read a fair number of term papers that have been created with it, and I can identify them pretty pretty easily now.

yeah, okay. So last question, big picture, does this totally change digital transformation as companies have previously thought about it?

I think it does, in the sense that, okay, you can have a small group of IT professionals doing your digital transformation, or you can have, I don't know, fifty, sixty, eighty of your people working at it. Obviously, you're gonna get a much faster transformation if, um, you have that democratized approach. And, you know, we don't have too many examples yet of really, you know, that broad scale adoption, but it is happening at that vendor that I mentioned previously.

I think it's gonna be happening at BMW when they've finished training their 80,000 people in it. So, um, I think the new digital transformation is gonna have to be a lot more democratized than the than the previous version was.

Well, terrific, Tom, thank you. I came into this conversation a little bit scared about having to become a programmer, but it sounds like really exciting development for all organizations. So I'll I'll get on board. Thanks so much for talking to me today.

Thank you.