A Brief History of AI

Turing’s Question and the Name AI

In 1950, the British mathematician Alan Turing published a paper that opened with a question: Can machines think?

Turing considered the question too vague. What exactly is a machine, and what does it mean to think? You can debate that for a long time without ever reaching a testable answer. So he replaced the question with a game.

In Turing’s original imitation game, an interrogator exchanged written messages with a man and a woman. The interrogator had to work out which one was the man. Turing then asked the crucial question: What happens if a machine takes the man’s place?

Today, the game is usually described in a simpler form. You exchange messages with a person and a machine without seeing either one. Your task is to identify which answers came from the machine. Turing did not define a universal point at which every machine would have passed the test forever. But he did make a specific prediction. By around the year 2000, after five minutes of questioning, an average interrogator should be right no more than 70 percent of the time.

This later became known as the Turing test. It asks whether a machine’s use of language in this game can be distinguished from a human’s. The result tells us nothing about consciousness or inner experience. Turing’s own approach was more cautious: he replaced a question that was hard to define with a task that could be observed.

Five years after Turing’s paper, the proposed field of research got its name. In 1955, John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon applied for funding for a summer project at Dartmouth College. The meeting was to take place in 1956. In their proposal, they called the field artificial intelligence, or AI.

Their idea was simple and fairly bold. Describe how learning and thinking work precisely enough, and you can teach a machine to do them too. Language, abstraction, problem-solving, neural networks, and machines that improve themselves were all on the list. And one summer with a few good people was supposed to be enough to make substantial progress.

The summer of 1956 did not produce a thinking machine. The participants arrived at different times, worked on different questions, and published no joint final report. What remained were the name and several lines of research that would occupy the field for decades.

Allen Newell and Herbert Simon presented the Logic Theorist, which they had developed with Cliff Shaw. The program proved theorems from Whitehead and Russell’s Principia Mathematica. For one of them, it found a shorter proof than the published version. The Logic Theorist is therefore regarded as an early automated theorem prover and an important program in symbolic AI.

Expectations rose quickly. In late 1957, Newell and Simon predicted that a computer would become world chess champion within ten years. We will return to that prediction later.

This chapter does not tell the complete history of every branch of AI. It follows one particular thread: from explicitly programmed rules to learning systems and then to language models. Along the way, a pattern emerges of great expectations, genuine progress, and setbacks.

ELIZA and Anthropomorphism

In the mid-1960s, Joseph Weizenbaum built a program called ELIZA at the Massachusetts Institute of Technology. A script called DOCTOR played the role of a psychotherapist. Here, a script means a collection of search patterns and response rules. ELIZA looked for keywords, broke sentences apart according to fixed patterns, and rearranged pieces of the input. “I am sad” could become “How long have you been sad?” “My father hates me” could prompt it to ask for more information about the father. ELIZA did not understand what these statements meant.

ELIZA was an example of symbolic AI, one of the field’s first major approaches. A symbol represents something unambiguously, such as “customer,” “invoice,” or “father.” Symbolic systems process these symbols using programmed rules. Search methods combine the permitted steps and can find solutions that nobody specified one by one. But people still define the representation, the rules, and the search method.

Weizenbaum was surprised by people’s reactions. His secretary had watched him work on the program and knew perfectly well that ELIZA was software. Even so, she asked him to leave the room so she could write to the program in private.

The phenomenon has since become known as the ELIZA effect: the tendency to attribute more understanding to a machine that uses language than its behavior actually demonstrates. Behind it lies an old human habit. We assume that something has an inner life as soon as it behaves like a person. The term for this is anthropomorphism.

The same tendency appeared again six decades later. In 2022, a Google engineer worked with the LaMDA language model and, after many conversations, declared that it was sentient. He wrote internal reports and later published transcripts of the conversations. Google rejected his assessment and eventually fired him.

ELIZA matched patterns. LaMDA was a language model. Their designs could hardly be more different, yet they created the same impression in people. Fluent language sounds like understanding, and some people even suspect consciousness behind it. This can happen even to those who know exactly how these systems work. We will look more closely at why the answers sound so convincing later.

The Bet on Rules

Symbolic AI did not stop at conversation programs. Rule-based systems became one of the first AI approaches to see practical and later commercial use. Such a system makes decisions using explicitly written if-then rules. ELIZA already used rules of this kind: if the sentence contains the word father, ask about the father. People write the rules, and the program applies them. A decision can usually be traced back to the rules that triggered it.

One early major bet on this approach was translation. During the Cold War, the United States wanted to translate Russian texts into English automatically. A dictionary and a few grammar rules seemed as if they might be enough. In January 1954, Georgetown University and IBM demonstrated an early version. The system translated about sixty carefully selected Russian sentences. The demonstration raised expectations that machine translation could become useful within a few years.

It became much harder with unrestricted text. Take the word “bank.” It can mean a place to sit or a financial institution, and only the context tells you which one is intended. The program needed another rule for every case like this, and new cases kept appearing. After twelve years and a great deal of money, there was still no program that could produce a usable translation of an arbitrary text.

The First Winter

Anyone providing that much money eventually wants to see results.

In 1966, a committee of the US National Academy of Sciences took stock. It examined performance, cost, and demand after twelve years of research. Its conclusion was sobering. High-quality, fully automatic translation of general texts was not in sight. The report recommended basic research and better tools for human translators. US government funding for machine translation was sharply reduced afterward.

Seven years later, James Lighthill submitted a review of AI research to the British Science Research Council. He acknowledged progress in individual areas but found that the far-reaching promises had not been fulfilled. The British government then cut support for AI research at most universities.

These cuts contributed to the first AI winter in the mid-1970s. During this period, funding, commercial interest, and public attention all declined substantially.

That did not mean research stopped. Work on neural networks, robotics, and statistical methods continued. What declined were funding and public expectations.

The reason for this collapse is worth examining. The translation programs of 1966 were better than the 1954 demonstration. But they were measured against the promise of fully automatic translation in the near future. Compared with that promise, the real progress looked small.

Expert Systems and the Second Winter

The grand idea had lost its funding, but work continued on a smaller scale. Instead of one machine for everything, researchers now built programs for a single field: expert systems.

An expert system also makes decisions with if-then rules. It is a more elaborate form of rule-based system. Its rules come from interviewing specialists and are meant to reproduce their judgment within a narrow field. The knowledge base contains facts and rules. A separate part of the program applies them to a specific case. The system can also explain which rules led to its result.

MYCIN at Stanford was one of the best-known examples. The research program was designed to advise physicians on choosing antibiotics for certain bacterial infections. A 1975 publication describes around 200 decision criteria.

One of these rules said, in essence: if the organism stains gram-positive, is spherical, and grows in chains, that supports a diagnosis of streptococci. The stain is a laboratory test that divides bacteria into two broad groups. The rule also had a certainty factor, in this case 0.7. The higher the value, the more strongly the finding supports the diagnosis. A single finding is rarely conclusive, so MYCIN combined these values across several rules. The system could also show which rules had led to a recommendation.

An evaluation published in 1979 is still cited today. For ten cases of meningitis, there was one recommendation from MYCIN and nine from people, including five specialists in infectious diseases. Eight other experts rated all the recommendations without knowing who or what had produced them. MYCIN’s recommendations were judged “acceptable” in 65 percent of the ratings. The five participating specialists scored between 42.5 and 62.5 percent. In this small retrospective comparison, MYCIN performed at least as well as the specialists involved.

Even so, it never became a clinical product. MYCIN remained a research system and was never used routinely to treat patients. The experiment had tested the quality of individual recommendations. Routine clinical use would also have required current patient data, integration into clinical workflows, further trials, and clear lines of responsibility.

The technology for a narrowly defined task can work even when the environment around it is not yet ready.

In the 1980s, the idea became a business. Companies commissioned expert systems for configuration, accounting, and troubleshooting. At the computer manufacturer Digital Equipment, a system called R1, later renamed XCON, assembled the right computer configuration for an order. At the same time, a market emerged for specialized Lisp machines. Lisp is a programming language that was widely used in AI research at the time.

By the end of the decade, the market had turned. Ordinary workstations became powerful enough, and the market for expensive Lisp machines collapsed. Expert systems also revealed how much work it took to collect thousands of rules, check them for conflicts, and keep them up to date. These developments contributed to the second AI winter.

Why Rules Remained

Rule-based systems themselves never disappeared. One use is calculating your car insurance premium. If you park the car in a garage at night, the premium goes down. If you drive for a year without an accident, you usually move into a better no-claims class and pay less. These rules are fixed in the policy, alongside statistical models for the risk of entire groups of drivers. A loan application works in a similar way. A bank might specify that the system should reject an application if the monthly payment exceeds forty percent of disposable income. Borderline cases may go to an employee instead. Legal requirements, credit bureau data, and risk models also play a role.

That alone does not make these programs AI. Knowledge-based systems are historically part of the field. Today, many of their basic ideas appear under names such as business rules, decision support, or rule engines.

For many tasks, fixed rules are the better choice. A deterministic rule-based system produces the same result from the same input. If someone asks, you can show which rules it applied. Both are more difficult with a language model. Depending on its settings, it may answer the same question differently. Its explanation also cannot be traced back to a single stored rule.

My Rule-Based Poker Bot

My poker bot worked the same way. Around 2010, I wrote a program that played online poker for me. The game was no-limit Texas Hold’em, the best-known variant. Each player receives two hidden cards, while five shared cards are gradually placed face up in the middle of the table. From those seven cards, you make your best five-card hand. No-limit means you may bet all the chips you have in any round.

I had read two strategy books, Harrington on Online Cash Games and Ryan Fee’s 6 Max NL Strategy Guide. I turned their recommendations into fixed rules. Two aces in your hand? Raise. Two low cards of different suits? Fold. The rules also covered your position at the table, meaning whether you acted early or late, the other players’ bets, and a few statistics such as how often an opponent had raised before. A few hundred rules and a bit of statistical analysis. That was all.

At low stakes, it ran reliably and made a profit over a large number of hands. It never became tired or changed its strategy out of frustration after losing a hand. It played the thousandth hand by the same rules as the first.

But the bot learned nothing from its games. As soon as an opponent played in an unusual way or the conditions changed, the right rule was simply missing, and I had to add it.

That is the limit. Every new exception requires a new rule, and a person has to resolve any conflicts. This still works well in a narrow field with clear cases. In an open world, maintenance grows faster than the benefit, and these rule-based systems reach their limits.

The Third Idea: Learning from Examples

So researchers asked a different question. What if the machine could find the rules itself?

Take a simple task. You have one hundred images. Fifty show a circle, and fifty show a square. A rule-based system would need instructions from you: count the corners, measure the curves, compare the edge lengths. You would have to adjust the rules for every tilted or blurry image.

A learning program, by contrast, receives only the images and the correct answer for each one. It guesses on the first image. If it is wrong, it adjusts itself slightly. Then comes the next image, another guess, and another adjustment. After many passes, it is usually right even though nobody told it anything about corners. Learning from examples in this way is called machine learning.

What changes are numbers inside the program. They are called weights, and they affect how information flows through the model. We will look at exactly how a learning step works in Chapter 8.

The idea itself was not new. Researchers had studied learning machines and artificial neurons even before Dartmouth. In 1958, Frank Rosenblatt described the perceptron, a single learning computational unit. It sorts examples into two groups, such as circles and squares.

Imagine the examples as points on a sheet of paper. A perceptron draws exactly one straight line and says: one group goes on the left, the other on the right. If the points can be separated by one line, it learns the task. If the groups are nested inside each other, one line is not enough.

For cases like that, a network needs an intermediate layer whose units each draw their own line. Every layer continues calculating with the output of the one before it. Something else has to happen between the layers to turn these into curved boundaries. Each unit passes its result through a function that bends it slightly. Without that bend, even ten layers together still produce only one straight line. This is called an activation function. With it, many straight lines can form boundaries of any shape. The whole structure is called a neural network. The name and basic idea were loosely inspired by the brain, but the mathematical units are highly simplified.

Human work has not disappeared. It has moved. Specialists used to write down rules. Now teams collect examples and decide what counts as the correct answer. They then choose the method that adjusts the weights. It is not easier, just different.

It would take several more decades before networks like these outperformed other methods on many practical tasks.

What Made the Breakthrough Possible

Three things had to come together before neural networks could solve many practical tasks: enough data, suitable compute, and better training methods.

A network learns from examples. For a long time, large collections of examples were missing. The internet changed that. Millions of people published images, videos, and text. Research groups could use them to build datasets that no single project could have created alone. The data still had to be selected, organized, and often labeled by people.

The second factor was compute. Ordinary computer processors also became faster, but graphics processing units were especially well suited to training neural networks. They had been developed for 3D graphics and could perform many similar calculations at the same time. That is useful for neural network training too, where a great many values are multiplied by weights and then added together. Hardware built for games became an important tool for AI research.

The third factor was better training methods. In a network with many layers, a great many weights contribute to the result. To adjust them in a targeted way, you have to calculate how each weight affected the error. Backpropagation does exactly that. It carries the error backward from the result through the network and shows how the weights need to change.

The basic idea was older. A paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams brought it to broad attention in the research community in 1986. Better activation functions, suitable initial values, and other improvements followed later. These made it possible to train deep networks more reliably.

In 2012, these three factors came together in the ImageNet competition. The task was easy to explain: a program received a photograph and had to say what it showed. A neural network won by a wide margin over the other methods. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton had trained it on graphics processors. The network became known as AlexNet.

AlexNet consisted of many consecutive layers. Machine learning with networks like this is called deep learning. The word deep refers to this sequence of layers.

With the rule-based systems from the first half of this chapter, a person had to write down how a program could recognize a dog. A neural network receives photographs with the correct answers instead and adjusts its weights from them. But it can also learn a pattern that has little to do with the actual task.

A company wants a system to identify defective parts in photographs, so it photographs its good and bad parts. The good parts are on the table in the morning, the bad ones in the afternoon. This means the lighting also differs between the images. The network may learn to distinguish bright light from dim light instead of looking for the defect in the part. It is simply following the examples it was given.

So where can you properly test whether a system like this is actually any good?

Games as a Benchmark

The easiest place to test this is where the result is unambiguous. Games offer exactly those conditions. Their rules are fixed, and the outcome is a win, a loss, or a draw.

An important early success came in backgammon. In 1992, Gerald Tesauro at IBM built a neural network for the game. In backgammon, you roll dice and move your pieces across a board, so chance plays a role in a way it does not in chess. An early version of TD-Gammon began without any preset rules for evaluating a position. The network played against itself and learned only from the results of those games. Later versions added a few human-defined features and a short search through possible moves. TD-Gammon reached a level close to the best players in the world. Top players even revised their assessment of some opening moves afterward.

Chess still attracted more attention. For centuries it had been associated with a sharp mind, and early AI research adopted that view. The idea was that mastering chess meant mastering thought. In late 1957, Newell and Simon had predicted that a computer would become world chess champion within ten years.

In fact, it took almost forty years. In 1997, IBM’s Deep Blue won a match against reigning world champion Garry Kasparov.

But the machine won in a different way from what Newell and Simon had expected. Its approach also differed greatly from the way TD-Gammon had learned five years earlier. Deep Blue examined a large number of possible move sequences in every position. It evaluated the resulting positions with a function developed by experts and used databases for openings and endgames. It did not learn from its games during the match. Its strength came mainly from fast search, programmed chess knowledge, and purpose-built hardware.

This approach reached its limits with Go. Every position offers so many possible moves that the search tree grows especially quickly. A Go position is also much harder to evaluate than a chess position. Professional players often described their decisions as a feel for the shape of the board. For a long time, Go was therefore considered a game in which machines would remain outmatched for decades.

Video games provided an important piece of the next advance. In 2015, DeepMind presented a program in Nature that learned to play 49 Atari games directly from the pixels on the screen and the score it received. The team used the same learning algorithm, network architecture, and settings for every game. In 29 of the 49 games, the program achieved more than 75 percent of the score of a professional human game tester. One method had learned very different tasks from the same kind of input and feedback. The same research group then applied this approach to Go and developed AlphaGo.

In 2016, AlphaGo played Lee Sedol, one of the strongest Go players in the world. The program had first learned from more than one hundred thousand human games. It then improved through a very large number of games against itself. In this kind of system, an agent tries actions and receives feedback about how successful they were. This form of learning is called reinforcement learning. AlphaGo still searched through possible sequences of moves, but its learned networks helped it focus on the most promising ones.

The second game became famous. On move 37, AlphaGo placed a stone where hardly any professional would have considered playing. The commentators initially thought the move was a mistake. AlphaGo won the game, and move 37 became one of the most discussed moments of the match. The network AlphaGo used to predict human moves had estimated the probability of that move at roughly one in ten thousand. The program had found a strong option that had barely appeared in its human training games.

In the fourth game, Lee Sedol won with an equally unusual move. It was his only victory. AlphaGo won the match 4 to 1.

AlphaGo Zero followed a year later. The new system received no human games and learned entirely by playing against itself. After three days of training, it won one hundred out of one hundred games against the version of AlphaGo that had beaten Lee Sedol. After 40 days, it was stronger than every earlier version. The comparison shows that human games were not necessary for this result. But the training data were not the only difference. DeepMind had also changed the algorithm and the network architecture.

The experiment then moved from the board to the screen. In the computer game Dota 2, two teams of five players compete to destroy the opposing team’s base. Each player controls a hero with distinct abilities. A match lasts around 45 minutes on average, and much of the map is covered by fog. A team can therefore see only what is happening near its own units.

In 2019, OpenAI Five defeated the reigning world champion team OG 2 to 0. The system then played publicly against human teams for four days. It won 7,215 of 7,257 games, although the human team had quit early in 3,140 of those wins. OpenAI Five also played a restricted version of Dota 2 with 17 heroes instead of more than one hundred.

For researchers, the nature of the task was especially interesting. A chess program makes a few dozen decisions per game. In Dota 2, it makes tens of thousands. Whether an individual decision was good often becomes clear only much later, from the outcome of the match. The system also has to deal with incomplete information and coordinate the actions of five heroes. The foundation was not a completely new learning principle. It was a known reinforcement learning method trained against itself at very large scale. We will return to this point at the end of the chapter.

What did games contribute to AI research? Above all, they provided an environment in which a system could evaluate its own performance and improve step by step. The rules are fixed. A game can be repeated as often as needed, and it produces a clear result at the end. This creates a loop of playing, evaluating, and adjusting. The loop was implemented differently in backgammon, Atari, Go, and Dota 2, but its basic principle remained the same.

Outside games, a result signal like this is often missing. A summary or a proposal has no natural score. People first have to decide what a good result even means. Anyone using AI in a company must therefore design this feedback themselves. Chapter 16 explains how. With open-ended language, this was precisely the next major obstacle.

2017: The Transformer

In 2017, eight researchers at Google published a paper titled “Attention Is All You Need.” It introduced a new architecture for neural networks: the Transformer. Until then, language processing had been dominated by recurrent neural networks. They processed the positions in a text one after another, with each calculation depending on the one before it. This slowed training and made it harder to connect words that were far apart.

The Transformer replaces this process with what is called an attention mechanism. For each position, the model calculates which other positions in the text matter at that moment. Take the sentence: “The cat that sat on the wall jumped down.” What jumped? The model can connect “jumped” directly to “cat,” even though several words stand between them. This weighting of relationships within a text is called self-attention. We will look more closely at the calculation behind it in Chapter 7.

One major advantage appeared during training. The Transformer can process many positions in an existing text sequence at the same time. These calculations can be spread efficiently across graphics processors. Researchers could therefore train larger models on more text. The sequence still matters when the model generates new text. It continues to write one small piece at a time. These pieces of text are called tokens. GPT and many later large language models are based on the Transformer or a variation of it. The T in GPT also stands for Transformer.

But the difference between the 2017 paper and later language models is more than size alone. The smaller base version of the original Transformer had 65 million weights, also called parameters. Models published later reached hundreds of billions of parameters. The amount of training data and the required compute grew at the same time. Researchers also improved many details of the architecture. In today’s assistant systems, the initial training is usually followed by several stages of post-training. Post-training is the umbrella term for every training step that follows. One of these is fine-tuning, meaning targeted additional training with new examples. These methods help a model follow instructions and produce more suitable answers.

2020: Three Examples Without Fine-Tuning

These larger models could do something that had previously required a project of its own. Imagine that two hundred support requests arrive in your inbox every morning. Shipping, accounting, technical support. Each one needs to reach the right department. It is tedious work that a program could take over.

Until 2020, this usually required fine-tuning. You collected a few thousand old requests, labeled each one with the correct department, and continued training the model on those examples. Its weights changed until it reproduced your sorting. That required data, compute, and someone who knew how to do it. It was too much for a quick experiment.

In 2020, OpenAI’s GPT-3 showed that there was often another way. GPT-3 was a Transformer like the one from 2017, only much larger and trained on much more text. You simply entered three completed examples, one after another. “Package has not arrived” belongs to shipping. “Invoice charged twice” belongs to accounting. “Forgot my password” belongs to technical support. Below them, you added the new request and left the department blank. The model continued the pattern.

The model itself remains unchanged. The three examples exist only in your request, and its weights are the same as before. This behavior is called in-context learning. Delete the examples, and the next request starts again without them.

For practical use, that was the real leap. An entire project involving laborious data collection and a training run became a few lines of text and a few minutes of work. Chapter 14 explains how to construct examples and instructions like these deliberately.

November 2022: The Technology Gets a Text Box

Until 2022, large language models were used mainly in research projects and individual applications. Anyone who wanted to use GPT-3 freely for their own tasks generally needed access to an application programming interface or a technical testing interface.

On November 30, 2022, OpenAI released ChatGPT as a research preview. Users were invited to try the system and provide feedback on its strengths, weaknesses, and problematic answers. The interface was essentially a simple chat window.

A model from the GPT-3.5 series ran behind that chat window. According to OpenAI, the underlying model had completed its training in early 2022. The Transformer itself was already five years old by then. But OpenAI had continued training the model specifically for conversation.

In the first step, human trainers wrote sample conversations. They played both roles: the user and the assistant. OpenAI then continued training the model on these conversations. This method is called supervised fine-tuning, or SFT.

Next, ChatGPT generated several possible answers to the same message. Human trainers ranked them from best to worst. OpenAI used these rankings to train a reward model, which was meant to predict which answer the trainers would prefer. OpenAI then continued training ChatGPT with this signal. The method is called reinforcement learning from human feedback, or RLHF. Both steps are part of post-training, which we will examine more closely in Chapter 9.

Post-training shaped the model’s response behavior. The chat window made it easy to access. You could write in everyday language, ask follow-up questions, and continue working on the result through conversation.

On December 5, 2022, five days after the release, OpenAI CEO Sam Altman wrote that ChatGPT had more than one million users. For January 2023, UBS estimated around one hundred million monthly active users based on Similarweb data. By that estimate, ChatGPT was the fastest-growing consumer internet application to date. Over the following year, other major AI providers launched their own public chat services.

AI had long been present in search engines, recommendation systems, maps, spam filters, and voice assistants. With ChatGPT, millions of people could easily have a direct conversation with generative AI. A long technical development became visible in a simple chat window. Developers then began expanding what a model could do while producing an answer.

After 2022: The Model Gets Time and Tools

Development moved quickly after ChatGPT. Four advances matter for the rest of this book. The first leads back to games, which offered something that open-ended tasks lack: an unambiguous result.

Language models generate their answers step by step. But with a calculation or a programming error, the first approach often leads in the wrong direction. Newer models therefore use additional compute before giving the final answer. Internally, they can produce intermediate steps, test approaches, discard them, and start again. These systems are called reasoning models.

Tasks with verifiable solutions are especially useful for training and evaluating this behavior. A calculation is either correct or it is not. A program runs or it crashes. A letter offers no equally clear signal. This brings back the same property that made games so valuable to AI research. OpenAI introduced o1-preview in September 2024. In January 2025, DeepSeek released the R1 model with downloadable weights and a technical report on its training. Chapter 10 explains how reasoning models work in more detail.

The second development concerned access. It was possible to run models on your own hardware before 2023, but the range of capable models available for this purpose grew substantially from 2023 onward. In February, Meta gave selected researchers access to the LLaMA models, although only for noncommercial research. Llama 2 followed in July under its own license for research and most commercial uses. Many more downloadable models appeared afterward. At the same time, many providers lowered their prices for processing text. Both developments affect who can use this technology and where their data resides. Chapter 20 covers that subject.

The third development extended language models beyond text. In March 2023, OpenAI described GPT-4 as a model that could process images as well as text. In September, the company initially introduced image and voice features for ChatGPT Plus and Enterprise. Many other products and models later offered similar capabilities. In current applications, you can upload a photograph of an invoice and ask questions about it.

In the fourth development, the application around the model receives tools rather than the model itself. The model can request an internet search, the execution of program code, or an action on a computer. The application carries out that step and returns the result to the model. Systems that plan several steps in this way and select suitable tools are often called agents. We cover the subject in more detail in Chapter 17.

The Pattern: The Bitter Lesson

This brings us almost to the present. One question remains: Is there a pattern in this history?

Imagine two chess programs. One contains a thousand rules written by grandmasters. The other knows only the rules of the game and learns from its matches. At first, the first program may be stronger. With more experience, the second can continue to improve. People have to add rules to the first program whenever it encounters a new situation.

The difference is how the two programs use additional compute. Written chess rules do not automatically improve when the computer becomes faster. A suitable search or learning method, on the other hand, can use more compute to examine more positions or learn from more experience.

Rich Sutton described this pattern in a short 2019 essay titled The Bitter Lesson. He looked back over seventy years of AI research. His observation was that general methods capable of using additional compute for search and learning tended to outperform solutions with extensive built-in expert knowledge in the long run.

Sutton calls the lesson bitter because built-in expert knowledge produces visible progress in the short term, so research teams invest a great deal of work in it. Over time, however, more general methods that can take advantage of greater compute often overtake those solutions.

Many stages in this chapter fit that pattern. The rule-based translation programs of the 1960s reached their limits. The market for expert systems shrank partly because the systems were so difficult to maintain. Deep Blue searched extremely quickly but retained evaluation rules developed by people.

TD-Gammon and AlexNet learned important features from experience and data. AlphaGo combined learned evaluation with search. AlphaGo Zero dispensed with human games and became substantially stronger. Language models learn statistical patterns from text instead of receiving a separate grammar program for every task or language.

The lesson is not a law of nature. Search needs a space of permitted possibilities. Learning needs data or generated experience. Reinforcement learning needs feedback that distinguishes good behavior from bad. People define these conditions. Explicit rules also remain necessary for approval thresholds, access rights, and laws.

For everyday use, the lesson is most useful as a question. Can the method make sensible use of more suitable data, experience, and compute? Or does someone have to add a rule for every new case? The answer tells you whether the solution can continue to improve with more data and compute.

Does the Bitter Lesson Apply to Language Models Too?

At first, language models look like strong evidence for the lesson. Training them to predict the next piece of text, combined with vast amounts of text and compute, displaced many hand-built language systems. Larger training runs reduced the error without anyone writing new grammar rules.

In 2025, Sutton offered a more nuanced view of language models. They use enormous amounts of compute, making them partly a classic example of his lesson. At the same time, they absorb large amounts of human knowledge from text, and those data are limited. During normal use, most language models also do not continuously update their weights through new experience. Sutton therefore expects systems that learn continuously from experience to scale further over the long term.

One counterargument is that imitation and experience need not be mutually exclusive. Reasoning models demonstrate part of this idea. They begin with a language model that has already been trained. During subsequent post-training, they can learn through reinforcement learning on verifiable tasks. But this is still not continuous learning from experience during later use.

The first published version of AlphaGo also began with human games and then improved by playing against itself. AlphaGo Zero later showed that it could succeed without this starting point in the clearly bounded game of Go. Whether that result can be transferred to open-ended tasks is still unclear.

It is also unclear whether today’s language models are the right starting point for systems like these. But the direction of research is clear. Models are meant to act, check results, and learn from the consequences of their actions.

What Happens Next?

The question is whether this time is different or another AI winter lies ahead. We have seen several waves: symbolic AI with its expert systems, followed later by the rise of neural networks. Their importance fluctuated sharply over the decades. Expectations repeatedly ran far ahead of what was practical. Periods later known as AI winters followed in the 1970s and from the late 1980s onward. This history leaves open whether today’s progress will continue.

Perhaps today’s wave is simply the next in that sequence. Three things do distinguish it from the earlier ones.

Direct access. Earlier AI successes often remained inside laboratories or narrow industries. Other systems worked invisibly in the background. With ChatGPT and similar applications, millions of people could work directly with a versatile model within a few months. That level of use produces feedback quickly and attracts investment.

Breadth. Earlier successes were usually specialists, with one program for each task. The same base model can now draft a letter, review program code, translate, and summarize text. How well it succeeds still depends on the task, the context, and the checks applied to its output.

Predictability of training. During earlier waves, it was difficult to foresee what a larger system would deliver. Under comparable training conditions, the development is more predictable today. When model size, the amount of data, and compute increase, a mathematical error metric falls according to predictable patterns. These relationships are called scaling laws. They are one reason companies can plan large and expensive training runs.

The error metric mainly shows how well the model predicts the next piece of text. The curves do not reveal which capabilities will emerge, how reliable the answers will be, or whether the model will be economically worthwhile. Chapter 8 examines this more closely.

In 1957, Newell and Simon gave the machine ten years to become world chess champion. It actually took almost forty. In the 2012 ImageNet competition, the error rate on the decisive metric fell from 26.2 percent for the second-best entry to 15.3 percent for AlexNet. The history of AI contains both inflated timelines and sudden leaps in performance. Together, they show how difficult it is to predict when a technical breakthrough will happen.

We have now reached the present. The term AI has referred to very different things in this chapter: ELIZA’s search patterns, an insurance company’s pricing rules, a network that sorts photographs, and finally a language model. The next step is to separate these things. Chapter 3 organizes them and shows where the language model fits.