He said that, based on talks with Uber’s senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features.
“That link is not there yet, right?” he said. “I think maybe implicitly there is more that is getting shipped, but it’s very hard to draw a line between one of those stats and, ‘Okay, now we’re actually producing 25% more useful consumer features.'”
Uber’s COO Says It’s Getting Harder to Justify the Money Spent on AI – Business Insider
On one hand, I kinda smirk at the idea of Uber getting caught up in the SAAS “sell it for super cheap and at a loss to destroy the competition and hook the buyer and then kick up those prices” business model that, well, made Uber.
On the other, this makes total sense when you consider the costs of AI at scale for companies like Uber, Duolingo, Microsoft, etc. Many companies have embraced AI and have pushed staff to use it just because it’s there.
Duolingo, for instance, walked back its decision to include AI usage in performance reviews after employees asked whether they had to use AI for the sake of using it.
“It felt like, rather than being held accountable for the actual outcome, we were trying to just push something that in some cases did not fit,” Duolingo CEO Luis von Ahn said in a podcast interview in April.
For a person using it at home on the free level generating funny lyrics or images, AI is a neat little toy. For companies looking to use it across the company for coding or as AI assistants, every little ask adds up in ways we’re not trained to look for. I mean, how much does it cost to Google something?
Add on the seemingly random token consumption* newer models of Claude or ChatGPT require to do even rudimentary tasks and it gets real tough to see the bang for buck.
* I say “seemingly random” because the use makes sense. In some ways the new models are pulling back the curtain on how they’re thinking behind the scenes, in others it’s tweaks that make them work better. But there’s no real method of measuring exactly what a query is going to cost until you commit, and when you’re paying by the token for hundreds or thousands of employees, that can run away from you real fast.
I’m curious if we have any real examples yet of AI usage seeing a net positive ROI across all players – company using AI sees profits from its use, AI provider is making enough money to more than cover its costs, etc.