Editing Openai/69408af5-1bf8-800f-b5dc-86cd01e07ec0 (section)

=== Assistant: Among the five main large language models LLMs often compared e.g., GPT-4, Google’s PaLM family, Anthropic’s Claude, Me… ===
Among the five main large language models (LLMs) often compared (e.g., GPT-4, Google’s PaLM family, Anthropic’s Claude, Meta’s LLaMA family, and Falcon), the one trained on the largest amount of data that’s been openly reported is:

Falcon-180B – trained on over 3.5 trillion tokens of text (the largest openly documented pretraining dataset among major LLMs). arXiv<ref>{{cite web|title=arXiv|url=https://arxiv.org/pdf/2311.16867|publisher=arxiv.org|access-date=2025-12-16}}</ref>

Comparative context (based on available public estimates):
* Falcon-180B (Technology Innovation Institute): trained on ~3.5 trillion tokens. arXiv<ref>{{cite web|title=arXiv|url=https://arxiv.org/pdf/2311.16867|publisher=arxiv.org|access-date=2025-12-16}}</ref>
* Google PaLM 2: reported trained on ~3.6 trillion tokens for some versions, though details vary and specifics can depend on which variant is referenced. Wikipedia<ref>{{cite web|title=Wikipedia|url=https://en.wikipedia.org/wiki/PaLM|publisher=en.wikipedia.org|access-date=2025-12-16}}</ref>
* Meta LLaMA 3.1: reported by some sources as pre-trained on ~15 trillion tokens (if including extended datasets), though this isn’t always officially disclosed in precise token counts by the developers themselves. Dan Taylor-Watt<ref>{{cite web|title=Dan Taylor-Watt|url=https://dantaylorwatt.substack.com/p/how-much-text-are-large-language|publisher=Dan Taylor-Watt|access-date=2025-12-16}}</ref>
* OpenAI GPT-4: number of training tokens isn’t publicly disclosed by OpenAI with precision; estimates exist but aren’t official. OpenAI<ref>{{cite web|title=OpenAI|url=https://openai.com/index/gpt-4-research/|publisher=openai.com|access-date=2025-12-16}}</ref>
* Anthropic Claude: earlier Claude models were reported trained on hundreds of billions of tokens, with newer versions less transparent about exact token counts. Wikipedia<ref>{{cite web|title=Wikipedia|url=https://en.wikipedia.org/wiki/List_of_large_language_models|publisher=en.wikipedia.org|access-date=2025-12-16}}</ref>

Conclusion: Of the models with publicly documented training data sizes, Falcon-180B has the largest confirmed training dataset. If relying on reported estimates for LLaMA 3.1 (~15 trillion tokens), LLaMA 3.1 might exceed that, but that figure isn’t officially disclosed in detail by Meta.