Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever.

Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind.

Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control

— I. J. Good, “Speculations Concerning the First Ultraintelligent Machine”, Advances in Computers, vol. 6, 1965

You know what I would love right now? Some eggs. I’m craving eggs. Why you ask? Well, I’ve just realized, as of writing this, that I am extremely hungry and my mind has conjured a delicious omelet as the perfect answer to my hunger. By now, my mind has fixated on that flavorful and soft omelet and my mouth has started to water. I open my beautiful handmade wooden box to find some eggs and lo and behold, I have no eggs. Zero. Zilch. What do I do, you ask?

Well, since you have kindly enquired, I’ve decided to come to you and ask you for some eggs. You have, in this imaginary hypothetical, an abundance of eggs, because you keep a lot of chickens. But you ask a question that is at the core of some of the topics we’ll touch today, for you ask,

“What can you give me for what you want?”

Now, that stumps me. Why? I have the ability to only do and make one single thing. I can only make boxes. I can make wooden boxes. These are boxes I have made by careful practice over the years, honed through repetition, creativity and hard work. I go pick the most beautiful box I’ve ever made throughout the years and I walk over to you and ask

“Say, you have a lot of eggs, right? You’ll need a good box to hold them. Here’s my best box in exchange for those eggs.”

You look at me and point to a dark spot in your house and say

“See that black box? That’s a box that makes me as many boxes as I need. I can vary size, vary features, specify and customize. I have no need of your boxes. You can’t give me anything I need.”

A naive, simple take, this harkens back to the era of barter. You had something, I needed something we would trade. But that means I had to have something you needed. How did we solve this? I trade with something someone else needed and got you what you needed. When we needed to scale this beyond three people, we invented a piece of copper stamped with an authority guaranteeing that copper had the value of a box or eggs or anything else. Thus, we began a simple economy.

In this simple economy, you could be great at something and get what you need by giving that away. Else, you could do everything yourself too, but not everyone wanted to.

Now, we are in a global economy: Connected, a breathing beast, with multiple nations, trade and financial instruments that take humans years of study to understand.

But, the fundamental question still remains: if all you can make are boxes, and if there is a magical box that can make boxes faster, better and cheaper than you, then what value do you have?

This is a question that is on my mind and on many others. We are facing a changing, transformative period and people are worried. This isn’t the first time we’ve faced such a transformative time, nor will it be the last. Thus, this is my attempt to coalesce the ideas and theories of economy I have uncovered (which may not be comprehensive; if that’s what you are searching for, this essay is not for you), and add my on-ground thoughts and experience of what could be seen as the “Canary In The Coalmine” for this change: Software Engineering.

Now, as I mentioned before, these changes aren’t new or unseen. As we’ll see, we’ve had many transformative changes to the world, from the Industrial Revolution to office software changing clerical jobs, to spreadsheets changing accounting and finance, to many others on a smaller scale. But a few things are different this time around, such as the impact on cognitive work, the speed of change and how impactful and powerful the transition would be to the world. History has always shown us that transitory periods are ripe for upheaval and socio-political implications. Thus, when changes happen over even shorter periods, it becomes even more critical to understand and form frameworks for the change that is happening, pulling from past history and economic theories, as the effects may not just be ripples, but humungous waves.

Let us go into this exploration, without “doom-erism” or un-founded blind optimism and try to see reality as it is. You will not find certainty here, for I am no oracle, but I hope this helps form theories of possible futures (at least in the short term).

As software ate the world, I’d like to ask: what would happen if AI ate software creation?

As always, in trying to make sense of the changes that are happening, we need to rely on the wisdom of people who have been thinking about this for far longer than me. Let us go through some of the theories and principles that are critical to understand these changes, pulling from economic frameworks. These are not exhaustive, and as I am nowhere near a real economist, lack academic rigor, but give us a starting point to make sense of what could be.

Job != Task

This seems simple enough, but has far-reaching implications as we think of automation. A job is not a single task. A barista, for example, has a task of turning on the espresso machine to make an americano. But the job of the barista is a collection of several tasks, bundled together to generate an output. Similarly, software engineering is not a single activity. It involves a multitude of tactical and non-technical tasks, including scoping, debugging, coding, deployment and maintaining, as well as coordinating with other people, which to several engineers’ frustration and to their surprise, takes a large amount of their day-to-day time. This distinction is important as we think about automation and AI, as there are tasks that can be automated, made easier, or made faster to do, but these are not just the job.

For us to critically look at what changes, we have to understand the forces of automation on each task. To be clear, jobs are not just lists of tasks. They are an interconnected network of tasks, some of which have multiplicative effects.

This dynamic is formalized by Michael Kremer’s O-Ring Theory of Economic Development. Kremer models complex production as a multiplicative chain where the failure of a single step, famously named after the faulty O-ring that destroyed the Challenger space shuttle, reduces the value of the entire output dramatically, even to zero.

Alternatively, when a highly time-consuming task within that network, such as creating small feature changes via code, is automated, the time required to complete it would drastically decrease. The human worker does not disappear; rather, their effort is forcefully reallocated to the remaining non-automated tasks, like system design or architectural decision making. Because a bottleneck has been removed from the task bundle, an individual worker suddenly produces significantly more finished software in the same amount of time. This type of increase in individual efficiency is the exact mechanism that alters the fundamental economics of an industry.

Automation lowers cost of output

Economists Daron Acemoglu and Pascual Restrepo, in their 2018 paper “The Race between Man and Machine”, use the concepts of the Displacement Effect and the Productivity Effect to explain this dynamic mathematically.

If some tasks become automated, then what happens to the remaining tasks and output in general? There are two forces. The first is the Displacement Effect, where automation allows firms to produce the same output with less human labor, directly displacing workers from specific tasks. Second is the Productivity Effect, which makes production cheaper. This is tied to Jevons Paradox: if automation makes production cheaper, the resulting capital savings and increased economic activity can drive up demand for non-automated tasks or entirely new roles, allowing the economy to absorb more labor. These effects act simultaneously.

Acemoglu and Restrepo’s framework is incomplete without its third, and perhaps most vital, pillar: the Reinstatement Effect.

While automation displaces workers from routine tasks, the introduction of new technologies simultaneously creates entirely new, complex tasks that require human labor. The machine pushes us out of the old jobs, but builds a new room we have to step into. It is the economic counterbalance to displacement.

The final output of all these effects hinges a lot on “demand.”

Demand decides outcome

Whether automation creates or destroys labour depends on the eventual demand of the output.

Agriculture is a well-quoted example here. Quoting from Alex Imas’s excellent essay, What will be scarce:

Large-scale automation made farmers—and eventually factory farms—much more productive. Agricultural production boomed and prices fell. But because people can only eat so much, the share of income spent on food went down as people got richer, and workers moved to manufacturing and then to services. The simultaneous fall of prices and reallocation of labor to another sector led to the perhaps non-obvious result that the more productive, automated sector became a smaller share of the economy despite serving and producing more. The less productive sector (services) where costs had not fallen—and in fact have risen—became a larger part of the economy.

How does demand play into this? Well, demand is the lynchpin on how this change can go different ways. In the example above of agriculture, humans can only eat so much. There is a limit. People will buy food, and there is an upper limit, after which, as we’ve seen through history, as prices of the food output fall, humans use the saved money and spend it in other sectors.

Armed with this lattice of frameworks, let’s venture forth to look at the current landscape and on-ground view of software engineering.

The Infinite Slot Lever, AKA Being Claude-Pilled

Writing code has become fundamentally different. I view the days of painfully typing out syntax character by character as a bygone era, looking back at it with the same fascination a millennial might reserve for the screeching tones of dial-up internet. I’d compare how we created software before to the equivalent of making a clay figurine. Pre-LLMs, you needed to mix and mold the material to your desired form. This meant hundreds of pushes and prods, beautifully sculpting the clay into the exact dimensions you envisioned.

Now, coding is similar to using a broad brush to create that same figurine. This magical brush can execute any alteration you want, but a single swipe will transform the entire mound of clay or drastically alter a specific part. Some engineers like to use this brush for very small adjustments and build the figurine up iteratively (reducing the hundreds of steps we used to take down to tens). Others, and the industry in general, are lazy, so the brush is just swung without care until the result resembles what we intended. Maybe there are textures on the figure you never explicitly added. Maybe the structure is more stable than what you could build manually. But in essence, we have sacrificed precise control to wield a magical brush that gets us 90% of the way there.

“AI-pilled” software engineers now pull on the slot-machine lever of the coding agent of their choice and voila, out pops a plausible-looking answer. This is then used to verify if the output matches the behavior. If not, you have two choices: write another prompt and pull the lever again or manually fix the issue. You can guess which one any human would prefer.

Why are we considering coding here? As mentioned previously, coding, considered one of the most highly “exposed” jobs, would allow us to have a glimpse into what may be if we continue upwards in the exponential curve of AI. As someone on ground zero, I provide some observations and further, using a few of the principles of economic theory, provide possible future scenarios.

Ground View Of Agentic Coding Shift

  • Writing large amounts of code is now easy. To clarify, this doesn’t mean code is good or meets expectations. Just that, typing code is no longer a bottleneck on a day-to-day basis.
  • Planning and architectural decisions are more important than ever. Good architecture, frozen and executed by coding agents, is the only way to generate and maintain codebases.
  • Implementing well-documented and common patterns is easier, such as building a common UI paradigm or writing REST API endpoints. Second-order effect: differentiation is harder.
  • AI Slop is real, and you actively need to fight against this entropy of large codebases, or this leads to more bugs and lower codebase quality.
  • More time per day is spent on verification and debugging.
  • Code review is now a major bottleneck. Review is harder, as the number of changes is also larger, as well as the quantity of changes (for example, LOC).
  • Maintainability and reliability of software are a bigger concern as understanding may be abstracted away by coding agents.

These are but a few of the massive shifts happening in the day-to-day work of software engineering. But let us be clear, from our framework we know: coding is a task, not the job. It is a massive component, but it is not the entirety of software engineering. Furthermore, it is not fully “automated.” Software engineers must still guide the output. There are known pitfalls: security vulnerabilities, hallucinations, and spaghetti code, but from an economic perspective, these are simply the realities of using new tools of the trade.

If you are a health-conscious buyer, you demand the organic, perfectly cultivated oranges. But if you are starving, any passable orange will suffice. The market is currently hungry for output, and “good enough” AI-generated code is feeding it.

Thus, since the literal act of software creation is dramatically easier, we need to think through the cascading implications.

The Productivity Squeeze

As software engineering productivity skyrockets, what happens to the engineers’ time? Well, they take on more and more tasks. A common refrain echoing across Bay Area engineers right now is this: “I thought naively my work would decrease with AI, but I am working much longer and doing way more.”

Engineers are churning out more software. Ideas are turbocharged, driven by intense market pressure to adapt or die. Big Tech jobs are no longer the cushy, rest-and-vest roles they once were, and this compounds the drive to adopt coding agents and have faster implementations. Today, a single engineer can manage vastly more scope. This isn’t a full replacement yet because SWEs do much more than just code, and only a fraction of the total task bundle could even be considered automated. Writing code today is less like laying bricks and more like riding a Sandworm from Dune: you are hanging onto a massive, powerful force and trying to steer it without getting thrown off.

But reasoning from first principles: if one engineer can produce 2x the output, the immediate result isn’t necessarily firing that engineer. The first observable sign may be a freeze on new headcount. Maybe you simply don’t need to hire the next entry-level engineer.

We have some initial signs of this, although this is still ongoing work and investigation from economists. The Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence paper from Erik Brynjolfsson, Bharat Chandar, and Ruyu Chen finds that even when accounting for general corporate downsizing, employment for entry-level professionals (ages 22-25) in AI-exposed roles has fallen by 13 to 16 percent, indicating a targeted, structural reduction in early-career hiring, even accounting for other factors.

The Golden Goose and the Demand Cliff

If teams can produce vastly more software, and firms can sell that software, doesn’t that eventually mean more labor? Yes, but as established, everything hinges on demand.

Software has “eaten the world,” and historical demand has been insatiable. As the bottleneck of creation is removed, everything will become software. Because the cost of output has plummeted, we will see Jevons Paradox in full effect.

This will trigger a short-term labor absorption phase. Let us call this the “Golden Goose Period.” The sheer volume of new software applications being spun up will require more human oversight & technical understanding, leading to a temporary hike in hiring and job creation. GDP would rise as the economy digests this massive increase in software production.

However, like any boom driven by an underlying technological deflation, there may be a cliff approaching. This cliff represents the eventual tapering off of demand, where after this period of Golden eggs, we quickly and dramatically find that the market has been saturated. Surprisingly, this fall could be even larger as companies hire more in the short term to keep up with the demand (and also due to current systems still needing supervision), but over time reach this point where (a) Coding Agents become even better and require even less supervision, and (b) demand dies down.

The Wage Shift

What happens to wages and entry-level hiring before we hit this cliff?

To understand where software engineering wages are heading, we have to look past the sheer volume of output and examine the nature of the tasks being automated. David Autor and Neil Thompson illuminated a critical mechanism in their article Beyond Job Displacement: How AI Could Reshape the Value of Human Expertise: automation does not affect all workers equally, and the wage impact depends entirely on whether the technology removes simple tasks or complex ones.

When automation removes an occupation’s simpler tasks, the work that remains is more complex, requiring workers to focus more on high expertise tasks. This winnows out less expert workers, but also raises the value of the expertise remaining, thus boosting wages. Conversely, when automation removes an occupation’s most complex tasks, the remaining work becomes less expert and thus more accessible to a greater number of workers. So even though the automation of expert tasks is productivity-enhancing, the influx of newly qualified workers tends to reduce wages, rather than boost them.

As software becomes cheap to create, tech companies will enter hyper-mode. Initially, these firms will pay competitively to retain top talent capable of steering the AI to build better, faster products. They are racing to capture market value.

But every software competitor, due to market pressure, would end up doing the same thing.

If everyone can instantly generate a digital box, software becomes commoditized. Margins will erode. When the product itself is perfectly commoditized, the bottleneck, and therefore the value, shifts elsewhere. If product creation is cheap, distribution may become the scarce resource. Talent compensation may shift toward marketing, sales, and aggressive go-to-market strategies as the primary sources of leverage. The companies will fight a race to the bottom on software pricing, and in this scenario, AI will have truly eaten the software engineering industry.

A Possible K Shaped Future

I predict and imagine a possible future scenario where early-career of SWEs trend downwards while demand for seniors increases. This increase is not constant, but a short-term trend till we hit the cliff.

This cliff is beyond the scope of this essay, but it is the point at which we would have a “country of geniuses in a datacenter.” If this point occurs, all lines (and this would impact major white-collar jobs) would trend downward.

What Might Be Scarce for Software Engineering Then?

If code is cheap, and software is infinite, what retains value for the individual engineer?

Agency

Even the most advanced “agentic” systems today still require an initial kick. They are highly capable & intelligent engines sitting in neutral. They require a human to identify the enemy, define the problem space, and point the machine in the right direction. The ability to identify the correct problem to solve, rather than just solving the problem, will become the scarce resource. This may change in the future, but as of the short-term, agents need humans.

Taste

Software creation is still an act that contains artistry. Taste matters, and if multiple software products fight each other on pricing, GTM and distribution, engineers can dramatically change the scope of software by knowing the levers to push on. One of them is creating products and experiences that produce joy and other positive emotions in humans, which still require understanding what makes humans happy, an area which firmly remains in the hands of humans.

Accountability

If things break, who owns them? Who fixes AI-slop bugs? Agents are not on the hook. When something in production or infrastructure goes down, accountability to own and fix relies on humans. Without humans-in-the-loop, you cannot have accountability.

The era of painstakingly sculpting clay is over. The sandworm is moving, and we should learn how to ride it.

Footnotes

  1. The second-order macroeconomic effects of this transition are vast and cannot be completely covered due to the unknown unknowns involved specifically in software engineering.

  2. I would consider this a critical must read to understand the simple question “What is scarce” in the future, focusing on the argument for relational nature of commodities by humans.

  3. It would be remiss not to highlight Baumol’s Cost Disease, which goes into this in much more detail.

  4. There is more to be said about where all the slot machines show up in modern society.

  5. You may already start to see this with many UI components generated by Claude having similar design or color palettes by default.

  6. Maybe not for too long, but current and future systems still need a “Goal.” And someone needs to assign this goal and verify.

  7. A required follow-up here is to explore how exactly “exposed” categories are computed.

  8. There is a steelman position to be made here: maybe other AI Agents require more software, keeping the demand up even after human demand tapers off. We have not modeled such a future yet.

  9. Or, if you prefer, Shoggoth.

Originally published on Haecceity.