More vs Better

Let’s pretend — for argument’s sake — that LLMs are a legitimate force multiplier for knowledge work.

I say ‘pretend’ here because I’m still on the fence. I’ve seen LLMs do AMAZING things in minutes that would take humans days or even weeks. And yet I’ve seen them struggle with the most basic tasks like counting ‘r’s in ‘strawberry’ or maintaining logical consistency across even a modest codebase.

So let’s pretend that appropriate harnessing exists to overcome their current limitations. What then follows from this?

If this was true, then LLMs could allow us to do MORE work, BETTER work, or ideally both.

In the software world, we are certainly seeing MORE code — the number of new repos and PRs on GitHub has climbed significantly in the last 18 months, explained largely by the wide release of LLMs.

But we don’t seem to be seeing software be getting any BETTER — at the same time, code quality across the industry seems to be diminishing and incidents increasing, with some studies showing LLM generated code introducing security vulnerabilities in 45% of cases and increasing risk up to 10x.

A similar logic is playing out with headcount … If LLMs were legitimate force multipliers, organisations would have the choice to use them to increase customer value — and therefore revenue — or cut costs by reducing headcount. Once you remove the Magnificent 7, SP500 earnings are up a meagre 4.1%.

But boy oh boy are there layoffs.

What conclusions can we draw from this?

Is it that LLMs can in fact be used for creating BETTER software and other knowledge work but organisations choose instead to optimise for more?

Is it that when given a new force multiplier like LLMs, most organisations choose cost cutting over market growth because they lack the vision or capacity for market growth?

Or is it just that LLMs currently lack the appropriate harnessing or capability to actually do work BETTER? That they are output multipliers, not quality multipliers?