The Evolution of Software: From Code to LLM Operating Systems

Overview

This course explores the fundamental shift in how software is created, moving from traditional manual coding to the era of Large Language Models (LLMs). It examines the transition through three distinct paradigms—Software 1.0, 2.0, and 3.0—and analyzes the emerging infrastructure of AI. By understanding LLMs not just as chatbots but as a new type of "computer" or "operating system," developers and students can better navigate the rapidly changing landscape of AI Agents and software engineering.

Background & Context

The current era of software development is characterized by extreme volatility; as noted by Andrej Karpathy, a significant portion of AI advice becomes obsolete within six months, and many tools fail within 90 days. This volatility exists because we are witnessing a fundamental change in the nature of computation that hasn't occurred on this scale for 70 years.

Historically, software was a set of explicit instructions written by humans for computers to execute. However, the rise of neural networks and subsequently LLMs has shifted the "programming" burden from writing logic to tuning data and crafting prompts. This course provides the framework for understanding this transition, helping learners decide which paradigm (1.0, 2.0, or 3.0) is most appropriate for a given problem and how the "AI grid" is being built to support this new way of computing.

Core Concepts

Software 1.0: Explicit Programming

Software 1.0 refers to traditional computer code written by humans in languages like C++, Python, or Java. In this paradigm, the programmer writes explicit, step-by-step instructions that the computer follows exactly. If the programmer wants a specific outcome, they must manually define every logic gate, loop, and conditional statement.

For example, if you were performing sentiment classification in Software 1.0, you would write a series of Python rules—perhaps searching for specific keywords like "good" or "bad" and using a set of if/else statements to determine the sentiment of a sentence. This method is highly predictable and explicit but becomes incredibly complex and brittle when dealing with high-dimensional data like images or natural language.

Software 2.0: Neural Network Weights

Software 2.0 represents a shift from writing code to optimizing parameters. Instead of writing explicit instructions, developers create a neural network architecture and then use a dataset and an optimizer to "train" the network. The "code" in Software 2.0 is actually the weights of the neural network—the millions or billions of parameters that determine how the model processes information.

A primary example of this is AlexNet, a pioneering image recognizer. Instead of a human writing rules to identify "what a cat looks like" (e.g., "look for pointy ears"), the model is fed thousands of images of cats, and the optimizer adjusts the weights until the model can identify cats on its own. Hugging Face serves as the "GitHub of Software 2.0," providing a repository where these weights and model configurations are shared and versioned.

Software 3.0: Prompt-Based Programming

Software 3.0 is the most recent paradigm, where Large Language Models (LLMs) act as programmable computers. In this model, the "program" is the prompt, and the "programming language" is natural language (English). This is a fundamental change because it allows for the orchestration of complex tasks without needing to train a new model from scratch or write thousands of lines of explicit code.

In Software 3.0, a task like sentiment classification is handled via a "few-shot prompt." Instead of writing Python code or training a neural net, you provide the LLM with a few examples of the desired output and a set of instructions in English. The LLM then uses its internal weights to execute the task. This makes the development cycle incredibly fast, as the "code" is written in a native human language.

The "Eating the Stack" Phenomenon

The "eating the stack" phenomenon occurs when a newer, more capable software paradigm replaces the functionality of an older one. This was observed during the development of Tesla's Autopilot. Initially, the system relied heavily on C++ (Software 1.0) to handle tasks like stitching together images from multiple cameras across time.

As the neural networks (Software 2.0) grew in size and capability, the developers found that the neural networks could perform these tasks more effectively than the manual C++ code. Consequently, large portions of the C++ codebase were deleted and migrated into the neural network weights. This same process is now happening again, as Software 3.0 (LLMs) begins to replace both traditional code and fixed-function neural networks.

The AI Utility and the "Intelligence Brownout"

LLMs are increasingly functioning like a public utility, similar to electricity. AI labs (OpenAI, Google, Anthropic) act as power plants, spending massive Capital Expenditure (CapEx) to train models (building the "AI grid") and Operational Expenditure (OpEx) to serve that intelligence via APIs. Users pay for this "intelligence" through metered access, typically priced per million tokens.

Because these models are treated as utilities, users expect high uptime, low latency, and consistent quality. This creates a new vulnerability: when state-of-the-art LLMs experience downtime, the world experiences an "intelligence brownout." Because so many workflows are now dependent on these APIs, a service outage effectively makes the global digital ecosystem "dumber" until the "voltage" (the API availability) is restored.

LLMs as Operating Systems

Beyond being a utility, LLMs are evolving into a new kind of Operating System (OS). In this analogy:

• The LLM is the CPU (the central processing unit that does the thinking).
• The Context Window is the RAM/Memory (where the immediate information for the task is stored).
• Tool Use/Multimodality are the peripherals and drivers that allow the OS to interact with the outside world.

This OS model is evidenced by the way apps are now built. For instance, an AI-powered code editor like Cursor is an "app" that can be run on different "OSs" (GPT-4, Claude, or Gemini) via a simple dropdown menu, much like how VS Code can run on Windows, Mac, or Linux.

How It Works / Step-by-Step

Choosing the Programming Paradigm

When building a feature, a developer must now decide which of the three paradigms to use based on the pros and cons of each:

1. Evaluate the Task:

- Is it a rigid, rule-based task? $\rightarrow$ Use Software 1.0 (C++/Python).

- Is it a high-dimensional pattern recognition task (e.g., custom image classification)? $\rightarrow$ Use Software 2.0 (Train/Tune a Neural Net).

- Is it a reasoning, language, or orchestration task? $\rightarrow$ Use Software 3.0 (Prompting an LLM).

1. Implementation Path:

- For 1.0: Write explicit logic $\rightarrow$ Compile $\rightarrow$ Execute.

- For 2.0: Collect data $\rightarrow$ Define architecture $\rightarrow$ Run optimizer $\rightarrow$ Deploy weights.

- For 3.0: Craft prompt $\rightarrow$ Test with few-shot examples $\rightarrow$ Iterate on English instructions $\rightarrow$ Integrate via API.

1. Integration:

- Use tools like Open Router to act as a "transfer switch," allowing the developer to switch between different LLM providers (e.g., switching from GPT to Claude) without rewriting the application logic.

Real-World Examples & Use Cases

Case Study: Tesla Autopilot

• Initial State: Heavy reliance on C++ for image stitching and temporal synchronization.
• Transition: As neural networks became more powerful, the C++ code was deleted.
• Result: The Software 2.0 stack "ate" the Software 1.0 stack, leading to a more flexible and capable system.

Case Study: Image Generation (Flux)

• The Base Model: The Flux image generator represents a massive set of parameters (the "center point" in a model atlas).
• The "Git Commit": When a user tunes a LoRA (Low-Rank Adaptation) on top of Flux, they are essentially creating a "git commit" in the Software 2.0 space, creating a specialized version of the image generator for a specific style or subject.

Scenario: Modern App Development

Imagine building a customer support agent.

• Software 1.0 component: A Python script that connects to the database to fetch a user's order history.
• Software 3.0 component: An LLM prompt that takes that order history and writes a polite, empathetic response to the customer.
• The Orchestration: The developer uses an LLM as the "OS" to decide when to call the Python script (1.0) and how to format the final output (3.0).

Key Insights & Takeaways

• Fluency in all three paradigms is essential: Modern engineers must be comfortable moving between explicit code (1.0), weight optimization (2.0), and prompt engineering (3.0).
• English is a new programming language: The ability to precisely instruct an LLM in English is now a core technical skill.
• Software is becoming more malleable: Because Software 3.0 is based on natural language, the speed of iteration is orders of magnitude faster than traditional coding.
• The "AI Grid" is centralized: Due to the massive cost of compute, we are currently in a "1960s-style" era where compute is expensive and centralized in the cloud, making us "thin clients" using time-sharing.
• The "Linux" of LLMs: The Llama ecosystem is currently the closest approximation to an open-source alternative (like Linux) to the closed-source providers (like Windows/Mac).
• The "Fab" Analogy: Building LLMs is similar to semiconductor fabrication. Companies like Google (using TPUs) are like Intel (owning the fab), while companies only doing software are "fabless."

Common Pitfalls / What to Watch Out For

• Over-reliance on a single provider: Depending on one LLM provider exposes you to "intelligence brownouts." Use tools like Open Router to maintain redundancy.
• Using the wrong paradigm: Trying to solve a rigid logic problem with a prompt (3.0) can lead to hallucinations; some things are still better handled by explicit code (1.0).
• Ignoring the "Half-Life" of AI Advice: Be wary of specific tool recommendations. Because the field moves so fast, a tool that is "essential" today may be obsolete in 90 days. Focus on the fundamental paradigms rather than specific libraries.

Review Questions

1. Contrast Software 1.0 and Software 2.0. How does the method of "programming" differ between the two, and what is the equivalent of a "code repository" for each?
2. Explain the "Operating System" analogy for LLMs. Which parts of a traditional computer (CPU, RAM, etc.) correspond to the components of an LLM system?
3. Scenario: You are tasked with building a system that must perfectly calculate taxes based on a 500-page legal document and then email the result to a client. Which software paradigms would you use for the calculation phase versus the communication phase, and why?

Further Learning

• Prompt Engineering: Explore advanced few-shot and chain-of-thought prompting to master Software 3.0.
• Fine-tuning and LoRAs: Learn how to move from Software 3.0 (prompting) to Software 2.0 (tuning weights) for specialized tasks.
• Agentic Workflows: Study how LLMs can use tools (Software 1.0 functions) to perform autonomous tasks, effectively acting as the "OS" for a suite of tools.