Case Study: AT&T cuts call-center AI costs with small language models from H2O.ai
Key results
The challenge
AT&T handles 15 million customer calls annually, generating a vast volume of recorded, transcribed, and summarized interactions. Classifying these calls against an 80-label system using large models such as GPT-4 was computationally expensive and difficult to scale.
The solution
AT&T distilled large language models into three smaller fine-tuned open-source models, including H2O.ai's Danube (a compact 1.8B-parameter model fine-tuned with H2O LLM Studio). The final ensemble combined 10 categories from Llama, 20 from Danube, and 50 from a classifier.
The results, in context
The ensemble of small language models achieved 91% accuracy, closely matching the performance of the previous, more expensive solution. By combining categories across Llama, Danube, and a classifier, AT&T reduced costs to 35% of the previous solution, with Danube accounting for just 10%.