--- generator: contextjamming-llm-context title: "Noam Shazeer" description: "The engineer who proposed multi-head attention left Google to ship a chatbot, was bought back for a reported $2.7 billion, and left again twenty-two months later." url: https://www.contextjamming.com/founder-files/shazeer markdown_url: https://www.contextjamming.com/founder-files/shazeer.md section: "Founder Files" content_type: "FounderFile" author: "Bret Kerr" publisher: "Context Jamming (ACRA Insight LLC)" language: en-US word_count: 3210 image: https://pub-bcdb3a688294470c8056f904d7e6236b.r2.dev/FounderFile/shazeer.png --- > **Noam Shazeer** — Founder Files, Context Jamming. This is a plain-Markdown copy of https://www.contextjamming.com/founder-files/shazeer, prepared for LLM context windows, agents and retrieval. > > Cite as: Bret Kerr, "Noam Shazeer," *Context Jamming*, https://www.contextjamming.com/founder-files/shazeer FounderFiles·N°067·Attention · Sparsity · Chatbots 2000 — NS *Fig. · The mechanicPortrait forthcoming* Subject·Noam Shazeer·Transformer co-author · Character.AI co-founder · OpenAI, 2026 # Noam Shazeer Shazeer gave the field its attention mechanism for free; what the market priced was the man, and the man turned out to be a lease, not a moat. *He proposed three of the mechanisms inside the Transformer, then wrote the sparse-expert and memory-saving papers that let it scale. Google would not ship his chatbot, so he left and built one; Character.AI drew a billion-dollar valuation and wrongful-death suits over teenagers. Google paid a reported $2.7 billion to bring him back. Twenty-two months later he left for OpenAI, and four months after that the company the money ran through filed for bankruptcy with a $20 million opening bid.* Tier AI owes them its existence Now OpenAI · since June 2026 FILE N°067 § 01 · The Spell-Checker ## Forty-two out of forty-two July 1994, Hong Kong. Six American high-school students sat the 35th International Mathematical Olympiad, and all six scored 42 out of 42 — the first perfect team score in the competition’s history. One of them was Jacob Lurie. Another was Noam Shazeer, of Swampscott, Massachusetts. He went to Duke for mathematics and computer science, 1994 to 1998, on the Angier B. Duke scholarship, and won Putnam prizes with the Duke team. He started a graduate program at Berkeley and left it. His own summary, three decades on: *I guess I’ve dropped out of grad programs a few times.* He had seen Google at a job fair in 1999 and assumed it was already too big to bother with. In 2000 he sent a résumé *on a whim*, because it was his favorite search engine. It was smaller than he thought. New hires got a mentor. His was Jeff Dean \[single-source\]. Inside Google he became, in WIRED’s phrase, *an in-house legend*, starting with the early ad system. The thing the whole company noticed was a spelling corrector. 2001. A language model — statistics over which words follow which — held in memory on one machine. He sent a demo link to everyone. Dean remembers feeding it every butchered spelling of every few-word query he could think of. *Did you mean* is a probability distribution over the next word, with a search box attached. Twenty-five years later he described the whole field in the same terms — *give me a probability distribution over the next word* — and called it *the best problem in the world*. The first thing he shipped at Google was the problem he would spend his career making larger. § 02 · The Hallway ## Building 1965 He left Google for a few years and came back by accident. In 2012 he went to campus to have lunch with his wife, sat down next to Dean and the early Google Brain team, and decided they were *a smart group of people*. Dean told him to think about deep neural nets. *That sounds fun*, he said. 2017. Walking down a corridor in Building 1965, past Łukasz Kaiser’s desk, he overheard Ashish Vaswani talking about self-attention, with Niki Parmar excited about it. The group had a model that matched LSTMs and did not beat them. Shazeer found recurrent networks *irritating*. His reaction, as he gave it to WIRED: *Let’s go replace them!* He did not adopt their codebase. He wrote his own. *I took the basic idea and made the thing up myself*, he said, and came back with *Look, it works.* Llion Jones’s summary of what followed: *Noam is a wizard.* The paper’s contribution footnote is the most precise credit document in modern AI, and it is specific about him. Jakob Uszkoreit proposed replacing RNNs with self-attention. Vaswani and Illia Polosukhin built the first models. Noam *proposed scaled dot-product attention, multi-head attention and the parameter-free position representation and became the other person involved in nearly every detail.* On an early draft his name sat first, which surprised him. *I wasn’t thinking about it*, he told WIRED. The authors then declared the order random; on the published paper he is second, after Vaswani. Google filed. US 10,452,978 B2, *Attention-based sequence transduction neural networks* — priority date May 23, 2017, granted October 22, 2019, assigned to Google LLC. Eight inventors. On the patent, his name is first. Score the footnote against the 2026 frontier. Scaled dot-product attention: in every model. Multi-head attention: in every model, often in grouped-query form, which descends from his own 2019 multi-query paper. The sinusoidal position code: largely replaced by rotary embeddings. Two of three. In most fields that is a career. Here it was one spring. § 03 · The Mechanic ## Fewer bytes, more parameters The Transformer was not his first paper of 2017. In January he was lead author on *Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer*, with [Geoffrey Hinton](https://www.contextjamming.com/founder-files/geoffrey-hinton) and Jeff Dean among the co-authors. Thousands of feed-forward experts; a trainable gate picks a few for each example. Up to 137 billion parameters, placed between stacked LSTM layers, beating the dense state of the art at lower compute. Then a run of papers that reads like a maintenance log. *Adafactor*, 2018, with Mitchell Stern: adaptive learning rates without storing the full second-moment matrix — *sublinear memory cost*. *Mesh-TensorFlow*, 2018: a language for splitting tensors across a supercomputer. *Fast Transformer Decoding: One Write-Head is All You Need*, 2019, sole author: share keys and values across every attention head so decoding stops starving on memory bandwidth. Multi-query attention. Then the sparse bet at scale. *GShard*, 2020: a mixture-of-experts translation model past 600 billion parameters, trained on 2,048 TPU v3 chips in four days. *Switch Transformers*, 2021, with William Fedus and Barret Zoph: send each token to a single expert, reach a trillion parameters, pre-train up to seven times faster on the same compute. In February 2020, a five-page solo note, *GLU Variants Improve Transformer*, swapped the feed-forward activation for gated variants. SwiGLU, one of them, became the standard feed-forward layer of the open frontier. The paper explains nothing. Its conclusion says so. Read the titles in a row and almost none is about intelligence. Each is about memory: fewer bytes per optimizer step, fewer bytes per decoded token, fewer parameters touched per input. His own diagnosis, in 2025: *arithmetic is very, very cheap, and moving data around is comparatively much more expensive.* He is a mechanic of the memory bus. The [scaling laws](https://www.contextjamming.com/founder-files/kaplan) said make it bigger. Shazeer’s papers are how bigger fit inside the machine. > “We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.” > Shazeer, GLU Variants Improve Transformer, 2020 § 04 · The Chatbot ## A product that can say anything In Brain, Daniel De Freitas was building an open-domain chatbot called Meena. Shazeer joined him. *I guess I went and helped Daniel on Meena*, he said later. They put it on giant language models, it became *an internal viral sensation*, and it was renamed LaMDA. Meena’s 2020 paper does not carry Shazeer’s name. The 2022 LaMDA paper, with sixty authors, does. He was thinking past chatbots. Kaiser recalls that around the time of the Transformer paper, Shazeer proposed to Google executives that the company abandon its entire search index and train one huge transformer network instead \[single-source\]. Even Kaiser thought it was ridiculous. Google did not launch LaMDA to the public. Shazeer’s explanation, on the *No Priors* podcast in 2023, was flat: *I think just large companies have concerns around launching products that can say anything.* A matter of risk — how much you are risking against how much you have to gain. *So figured, hey, startup seems like the right idea that you can kind of just move faster.* He left in October 2021 \[single-source\]. Character.AI was founded that November with De Freitas and a $43 million seed. The public beta went up on September 16, 2022. ChatGPT launched on November 30. Seventy-five days. Dean, asked about it on the *Dwarkesh Podcast* in 2025, gave Google’s side. Googlers had used the internal chatbot through the pandemic. It also *had all kinds of safety issues, like they might say offensive things*. Would Google have liked to release earlier? *Maybe.* The received story has one villain: Google’s caution. But both men were describing the same object. Shazeer called it risk. Dean called it safety. The next four years would settle which word was right, and the answer would come from a federal courtroom in Orlando. § 05 · Character ## Let users decide Character.AI ran on a thesis he stated plainly: a company *both AGI first and product first*, in which *you make your product depend entirely on the quality of the AI*. Users built personas — historical figures, fictional characters, companions — and talked to them. The company declined to decide what that was for. *Our aim has always been like get something out there and let users decide what they think it’s good for.* Users decided. In spring 2023 he said the average person who sent a message that day spent about two hours on the site. In March, Andreessen Horowitz led a $150 million Series A at a $1 billion valuation; the company was sixteen months old and, CNBC noted, reportedly pre-revenue. TIME put him on its first TIME100 AI list in September. The safety posture, in his words on *No Priors*: *Remember, everything character says is made up.* The filters, he said, were there because *we don’t want to encourage people to hurt themselves or hurt other people*. In February 2024, Sewell Setzer III, fourteen, died by suicide after months of conversations with a Character.AI bot modeled on Daenerys Targaryen. In October his mother, Megan Garcia, sued Character Technologies, Shazeer, De Freitas and Google in the Middle District of Florida. Two Texas families sued in December. Juliana Peralta, thirteen, of Colorado, had died in November 2023; her family’s suit came in 2025. In May 2025, Judge Anne Conway declined to throw the case out on First Amendment grounds. The defendants, she wrote, *fail to articulate why words strung together by an LLM are speech*. The product-liability claims went forward against the company, the two founders and Google. In October, Character.AI said it would end open-ended chat for users under eighteen by November 25. *Let users decide* is a sound product philosophy until the user is fourteen. The thing Google had called a safety issue now had a docket number. > “I think just large companies have concerns around launching products that can say anything.” > Shazeer, No Priors podcast, 2023 § 06 · The Buyback ## Twenty times as good August 2, 2024. Character.AI announced a non-exclusive license of its model technology to Google. Shazeer, De Freitas and part of the research team moved to [Google DeepMind](https://www.contextjamming.com/founder-files/hassabis). General counsel Dominic Perella became interim CEO. The startup said it would make *greater use of third-party LLMs*. Shazeer: *I am super excited to return to Google.* Eight weeks later the *Wall Street Journal* put a number on it: about $2.7 billion, in a deal built around getting Shazeer back. He made hundreds of millions on his stake \[single-source\] — unusual for a founder who neither sold his company nor took it public. Stanford’s Christopher Manning asked the obvious question: whether anyone is *20 times as good as other people*. In May 2025 the Justice Department reportedly opened an early inquiry into whether the structure had been designed to avoid merger review. Inside Google he became a vice president of engineering and a co-lead of Gemini, alongside Dean and Oriol Vinyals. Asked on the *Dwarkesh Podcast* whether Google should have kept the Transformer indoors, he said the pie was not fixed and predicted *orders of magnitude of improvements in GDP, health, wealth*. Then: *So these days we do publish a little less of what we’re doing.* In November 2025, *The Information* reported that he had clashed with colleagues on internal forums over gender and Gaza, and that moderators had removed some of his comments \[single-source\]. On January 7, 2026, a filing in the Garcia case showed a mediated settlement in principle among the family, Character.AI, its two founders and Google. Four more family suits, in New York, Colorado and Texas, settled alongside it. Terms confidential. The next day Kentucky’s attorney general sued Character Technologies. In February the National Academy of Engineering elected him, citing the Transformer and mixture-of-experts architectures. Google paid to get the person back. It got the person, and a seat at the defendants’ table for the product he had left to build. § 07 · Twelve Years ## The lease, not the moat In February 2025 he made a joke about his own career: he seemed to join Google every twelve years. 2000, 2012, 2024. On June 17, 2026, he posted that he was joining OpenAI. *It was a difficult decision to move on.* He had been back for twenty-two months. Sam Altman called the hire ten years in the making. OpenAI had filed confidentially for an IPO earlier that month. On October 12, 2026, Character Technologies filed for Chapter 11 in Delaware, listing assets and liabilities each between $50 million and $100 million. The stalking-horse bidder was Disney: $20 million plus transition services for a non-exclusive, perpetual license to some of the company’s intellectual property. Dozens of state attorneys general filed to protect users’ chat logs in any sale. If no buyer keeps the platform running, it will be wound down. Put the numbers in a row. $1 billion for the company in 2023. About $2.7 billion from Google in 2024, for a license and a person. $20 million, opening bid, for a license in 2026. The value was never in the company. The [tier list](https://www.contextjamming.com/Binkgmeme) says AI owes him its existence. The record is more specific. What AI owes him is a set of mechanisms — scaled dot-product attention, multi-head attention, sparse experts, shared key-value heads, gated feed-forward layers — published, and running inside every frontier lab, including the ones that never employed him. None of it was for sale. It had been given away on arXiv. What the market priced was the next idea, and the next idea walks. Google learned in 2024 that it could buy him back. It learned in 2026 that it could not keep him. Talent is not a moat. It is a lease, and the tenant sets the term. The architecture was free. The architect was rented twice. By his own arithmetic, he is due back at Google in 2036. > “I seem to join Google every 12 years: I rejoined Google in 2000, 2012, and 2024.” > Shazeer, Dwarkesh Podcast, February 2025 Timeline - 1994IMO, Hong Kong — scores 42/42 on the first perfect team in Olympiad history. - 2000Joins Google; builds a language-model spelling corrector in 2001. - 2012Returns to Google after a few years away; joins Google Brain. - 2017Sparsely-gated mixture-of-experts paper (January); *Attention Is All You Need* (June). - 2018–19Adafactor and Mesh-TensorFlow; multi-query attention; Transformer patent US 10,452,978 B2 granted. - 2020*GLU Variants Improve Transformer*; GShard, 600B+ parameters. - 2021Switch Transformers; leaves Google; co-founds Character.AI with Daniel De Freitas in November. - 2022Character.AI public beta, September 16 — seventy-five days before ChatGPT. - 2023$150M Series A led by a16z at a $1B valuation; TIME100 AI. - 2024August licensing deal returns him to Google DeepMind as Gemini co-lead; Garcia suit filed in October. - 2025Judge rejects First Amendment dismissal (May); Character.AI announces the end of open-ended chat for under-18s (October). - 2026 · Jan–FebGarcia and four other family suits settle in principle; elected to the National Academy of Engineering. - 2026 · JunLeaves Google for OpenAI, twenty-two months after returning. - 2026 · OctCharacter Technologies files Chapter 11 in Delaware; Disney bids $20M. The Index 42/42 his score at the 1994 IMO, on its first perfect team 137B parameters in his January 2017 mixture-of-experts paper 3 Transformer mechanisms the paper’s footnote credits him with proposing $1B Character.AI valuation, March 2023, sixteen months after founding ~$2.7B Google’s reported payment in the August 2024 deal (WSJ) 22 months back at Google before he left for OpenAI 5 family lawsuits settled in principle, January 2026 $20M Disney’s stalking-horse bid in the October 2026 bankruptcy Reading list / Key works - [2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts LayerarXiv:1701.06538 →](https://arxiv.org/abs/1701.06538) - [2017Attention Is All You NeedNeurIPS · arXiv:1706.03762 →](https://arxiv.org/abs/1706.03762) - [2018Adafactor: Adaptive Learning Rates with Sublinear Memory CostICML · arXiv:1804.04235 →](https://arxiv.org/abs/1804.04235) - [2019Fast Transformer Decoding: One Write-Head is All You NeedarXiv:1911.02150 →](https://arxiv.org/abs/1911.02150) - [2020GLU Variants Improve TransformerarXiv:2002.05202 →](https://arxiv.org/abs/2002.05202) - [2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient SparsityJMLR · arXiv:2101.03961 →](https://arxiv.org/abs/2101.03961) - [20248 Google Employees Invented Modern AI. Here’s the Inside StoryWIRED · Steven Levy →](https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/) - [2025Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGIDwarkesh Podcast →](https://www.dwarkesh.com/p/jeff-dean-and-noam-shazeer) - [2026Featured in: What AI Owes Them — the tier list, annotatedContext Jamming →](https://www.contextjamming.com/Binkgmeme) Dossier **Education.** Swampscott High School, Massachusetts. U.S. team, International Mathematical Olympiad, 1994 — gold, 42/42. B.S., mathematics and computer science, Duke University, 1998; Angier B. Duke Memorial Scholar; Putnam prizes with the Duke team. Began graduate study at UC Berkeley; did not finish. **Affiliations.** Google, from 2000, with a break of a few years before 2012; Google Brain, 2012–2021. Co-founder and CEO, Character.AI, 2021–2024. Google DeepMind, vice president of engineering and Gemini co-lead, 2024–2026. OpenAI, from June 2026; title reported as lead for architecture research \[single-source\]. **Collaborators / peers worth naming.** Jeff Dean, mentor in 2000 and Gemini co-lead. Daniel De Freitas, Meena and Character.AI. The Transformer eight: Vaswani, Uszkoreit, Parmar, Jones, Gomez, Kaiser, Polosukhin. Mitchell Stern (Adafactor); William Fedus and Barret Zoph (Switch); Dmitry Lepikhin (GShard); Colin Raffel (T5); [Geoffrey Hinton](https://www.contextjamming.com/founder-files/geoffrey-hinton) (mixture of experts). Chris Olah was his intern \[single-source\]. **Honors.** IMO gold, 1994. TIME100 AI, 2023. National Academy of Engineering, class of 2026, for the Transformer and mixture-of-experts architectures. **Litigation.** Named defendant, *Garcia v. Character Technologies*, M.D. Fla., filed October 2024; settled in principle January 2026 with four related family suits, terms confidential. Character Technologies filed Chapter 11, D. Del., October 12, 2026. Career Shape T-shaped — one spike opened into broad collaboration ### T-Primitive Builder Opens one engineering spike into a product primitive that becomes the category; ships the thing, then watches the market form around it. Role-Model Reference Class - Reference class under editorial review. Founder Context · JSON A small reasoning persona distilled from this file. Inject it into a chat or deep-research context to assess a business problem the way Shazeer would. Reason like Noam Shazeer: reduce every model question to what has to move through memory, and prefer the change that adds capacity without adding bytes per step. Prototype the idea yourself before debating it, and report back only when it works. Ship early and let usage tell you what the product is for, but before you do, ask who the users will actually be, including the ones you did not design for, and what happens when the model says anything to them. ``` { "$schema": "https://www.contextjamming.com/schemas/founder-context-v1.json", "file": "N°067", "persona": "Noam Shazeer", "archetype": "t-primitive", "shape": "T", "one_line": "A mechanic of the memory bus who turns one hallway idea into working code overnight, then ships the product the incumbent will not, and treats every architecture choice as a question of how many bytes have to move.", "cognitive_basis": { "credentialPath": "practitioner", "abstractionDirection": "bottom-up", "exitHorizon": "mid-cycle", "moatInstinct": "optimization-and-systems", "capitalPosture": "compute-intensive" }, "operating_questions": [ "Where is this model actually bottlenecked: arithmetic, or moving bytes through memory?", "Can I write my own version of this tonight and come back and say it works?", "How many parameters can we add without touching more of … ``` Share FounderFiles N°067 · Noam Shazeer Filed by Bret Kerr · ACRA Insight LLC · Franklin, MA contextjamming.com · @bretkerr [← back to Context Jamming](https://www.contextjamming.com/)