Why this one paper matters
If you work in IT but not in AI, here is the one paper to understand. Everything people call "AI" today — ChatGPT, Claude, Gemini, the coding assistants, the image describers — runs on an idea introduced in a single 2017 paper from Google with a cheeky title: "Attention Is All You Need."
Eight researchers, eleven pages, and a design they called the Transformer. It didn't win a prize on release. It just quietly became the foundation everything else was built on. The "T" in GPT stands for Transformer.
This is that paper, explained without the maths — what problem it solved, the one idea at its heart, who built it, and why it changed the industry.
The problem: computers used to read like a slow reader
Before 2017, the best language software read a sentence the way a tired person reads aloud: one word at a time, left to right. These systems (called RNNs) processed word 1, then word 2, then word 3 — each step waiting for the one before it.
Two problems came from that:
It was slow. You couldn't do the words in parallel; step 5 needed step 4 first. That capped how big and how fast these models could get.
It forgot. By the end of a long sentence, the early words had faded. Ask it what "it" referred to twenty words back and it often lost the thread.
The idea: let every word look at every other word, all at once
The paper's move was to throw out the one-word-at-a-time chain entirely and replace it with a single mechanism: attention.
Attention is simpler than it sounds. For each word, the model asks: "which other words in this sentence should I pay attention to, to understand this one?" — and it does this for every word simultaneously, in parallel.
Think of a meeting where, instead of a message being whispered down a line of people, everyone can hear everyone at once and each person decides who to listen to. That's the shift.
Here's the classic example. Take the sentence "The animal didn't cross the street because it was tired." What does "it" refer to — the animal or the street? You know instantly. A word-by-word reader struggles. Attention lets the word "it" look back at every other word and put most of its focus on "animal":
The remarkable part: nobody programmed that rule. No engineer wrote "if the word is 'it', check for a nearby animal." The model learned those focus weights by reading enormous amounts of text. That is the whole trick — grammar and meaning emerge from the weights, not from hand-written rules.
Three small ideas that make it work
The paper adds three practical pieces on top of attention. In plain terms:
Multi-head attention. Rather than looking at the sentence one way, the model looks several ways at once — one "head" might track who did what, another what refers to what. Several perspectives, combined. (The paper uses eight.)
Positional encoding. If you look at all words at once, you lose their order — and "dog bites man" is not "man bites dog." So each word is tagged with its position, letting the model keep word order without reading in sequence.
Stacking. One layer of attention finds simple relationships; stack several (the paper uses six) and later layers build richer understanding on top of earlier ones — like re-reading a paragraph a few times, each pass catching more.
The whole machine, in one picture
Put it together and you get the Transformer: an encoder that reads and understands the input, and a decoder that writes the output one word at a time — each half built from stacked attention layers.
You do not need to trace every box. The shape is the point: read the input with attention, then write the output with attention, and do it in parallel instead of in a chain.
Who built it — and where they went
The paper came out of Google — specifically the Google Brain and Google Research teams — and was presented at NeurIPS 2017. Its eight authors are listed as equal contributors, which is unusual in itself.
What's more striking is what happened to them next. Almost every author left Google to go and build the AI industry their paper made possible:
Ashish Vaswani and Niki Parmar — co-founded Adept AI, and later Essential AI.
Noam Shazeer — co-founded Character.AI, then returned to Google DeepMind in 2024 to help lead Gemini.
Aidan Gomez — co-founded Cohere, an enterprise language-model company.
Llion Jones — co-founded Sakana AI in Tokyo.
Jakob Uszkoreit — co-founded Inceptive, applying the same ideas to biology and mRNA design.
Łukasz Kaiser — joined OpenAI, and worked on the models behind ChatGPT.
Illia Polosukhin — co-founded NEAR Protocol.
The pattern is the story: one eleven-page paper seeded a whole generation of AI companies. Journalists now profile the eight as a founding generation of the modern AI boom.
What happened next — the industry it built
The Transformer didn't stay an academic curiosity. Here's the chain reaction, in plain terms:
2018 — it took over language. Google's BERT and OpenAI's first GPT, both Transformers, swept the language benchmarks within a year.
2019–2020 — scaling worked. GPT-2 and then GPT-3 showed that simply making a Transformer bigger, on more text, kept making it smarter — a bet nobody could take for granted before.
November 2022 — it went mainstream. ChatGPT (a Transformer) reached an estimated 100 million users in about two months — the fastest a consumer app had ever grown to that point.
It spread far beyond text. The same attention mechanism now powers image models (Vision Transformers), speech, and even biology — DeepMind's protein-structure work uses attention.
It moved markets. Demand for the GPUs that train Transformers helped make Nvidia one of the most valuable companies on earth.
By the numbers. Attention Is All You Need has been cited well over 100,000 times — among the most-cited papers in all of modern science, and one of the fastest ever to get there.
The one-line takeaway
Stop reading word by word; let every word look at every other word at once, and learn what to focus on from data. That single change is what made modern AI possible — and it's why a paper called Attention Is All You Need turned out to be exactly right.
Reference: Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser & Polosukhin — "Attention Is All You Need." Google Brain / Google Research, 2017. Read the original: arxiv.org/abs/1706.03762 ↗. (This is a plain-English summary and commentary; all credit for the work belongs to the original authors. The diagrams here are my own, drawn to explain the ideas.)
Tags: transformers · attention · large language models · deep learning · foundational papers