Dell’s CAIO Dishes on Difficult Enterprise AI Journey

Many companies have struggled with their AI rollouts, including Dell Technologies, the $360 billion, Texas-based computer maker with nearly 100,000 employees. At the recent AI Infra Summit in Santa Clara, California, John Roese, Dell’s global CTO and chief AI officer, discussed his company’s struggles, and shared some tips on how it eventually found massive success.

“One of the things that maybe surprised us early on is how slow the enterprise adoption curve has been,” Roese told C. Vijayakumar, the CEO and managing director of HCLTech during a fireside chat at AI Infra Summit titled “The AI Velocity Equation: Balancing Speed, Sovereignty and Enterprise Advantage.”

“We learned a lot doing it to ourselves,” Roese continued. “When we started the journey, we realized that it was not just simply about the technology. To get an enterprise into production [with AI] is non-trivial. Many of the things you have to deal with have nothing to do with the technology. It started with governance.”

John Roese is the Global CTO and Chief AI Officer for Dell Technologies When Roese was put in charge of Dell’s AI strategy, the company had 900 AI projects scattered around the business, leading to chaos. The company made a decision to throw them all away and start fresh with four new AI projects in key areas of the business: sales, services, supply chain, and engineering.

“We had to change the culture of it from being a service center to a strategic resource,” Roese told his audience. “We had to retool the skill set. We went from random technology skills to AI pods that were a combination of basically a pair of programmers, TPMs [technical program managers], AI engineers, and data scientists working as an atomic unit. We’ve never done that before.”

One of the big hurdles that Dell had to overcome was understanding the specific business processes that it wanted to impact with AI. Like many large enterprises, Dell’s business processes were not necessarily written down and followed like computer code. Instead, the processes lived organically within the organization, requiring an extra step for understanding.

Another challenge was getting access to the hundreds of petabytes of data that powers Dell and understanding what it meant. “There’s no semantic layer, there’s no context layer,” Roese said. “And so we went on that journey as Dell.”

Developing the infrastructure was another challenge for Dell. The concept of “infrastructure” isn’t nailed down, and many things can be infrastructure, Roese said.

“Infrastructure in an enterprise is not a single thing,” he said. “You are applying AI to the processes and structures that make your business work. A typical company like either of ours might have a million processes that actually make it work. Some of them are extremely high value. Some of them are fairly low value. Some of them are incredibly sensitive. Some of them you don’t care if people see the data.”

Having a flexible, hybrid AI infrastructure that adapted to the different business processes and their various locations, SLAs, importance, and security rules was a critical element of Dell’s eventually success, Roese said.

For example, one of Dell’s AI projects is an agent that cleans up CRM database records autonomously without human intervention. The value of each database record that these agents cleans up is small, perhaps $0.50 per record, Roese said. Matching that agentic job to the cost of running the infrastructure was critical.

“I have millions of CRM records, and if I clean them all up, that has value,” Roese said. “But each transaction is very low atomic unit value. So if I build that agent and I use $12 in tokens to do $0.50 worth of work, I go out of business.”

Dell made this particular agent work by using private infrastructure, open models, and various scheduling and batching tricks to drive the cost down so that it matches the economic value of that work.

“At the other end of the spectrum, I have a set of agents that support my bosses, the founder of the company, and people, and those agents help them do their work,” Roese said. “They primarily gather and organize information so they can make immediate decisions on things. The reality is, I am perfectly comfortable spending thousands of dollars in tokens for that outcome, because the strategic value of it is entirely different than that first agent.”

Today, Roese has about 5,000 internal customers at Dell who are “consuming GPUs,” (i.e. doing AI work). The jobs range from the low-level database record agent to the reasoning agents supporting Michael Dell, and everything in between. The return on investment of the projects range from 20 to 1 to 30 to 1, which is “significant impact,” he said.

(DCStudio/Shutterstock) Over the past two years, Dell has grown revenues by $30 billion while lowering costs by 7%, which represented the first time ever the company had decoupled revenue growth from costs. Roese attributes at least some of that financial improvement to the AI projects.

“My observation is the enterprise is on a journey,” he said. “The journey is much longer than just simply building a model or standing up an AI technology or an infrastructure. It’s actually quite hard because it has organizational challenges, sociological challenges, cultural challenges, and technology challenges.”

While Dell is finding success, the effort to get there was non-trivial. When Roese started the AI initiative, Roese and his team had to build their own infrastructure, all the software, and all the tools.

“I got open source and I had to hack things together and it was very slow,” he said. “Today we’ve industrialized that and I just buy agented platforms. I’m able to access models in many different places, and it manifests in a very interesting metric.”

When Roese started the project, it took Dell seven and a half months to go from idea to production. Today it’s less than four weeks.

“And it’s not because the ideas became simpler,” he said. “It’s because the infrastructure is calcifying around standardized technology protocols, interfaces. When that happens, it allows the second wave of customers to benefit from that.”

Dell’s CAIO Dishes on Difficult Enterprise AI Journey

Comments