Read papers. Together.

"passes a simulated bar exam with a score around the top 10% of test takers"

The 'simulated' qualifier is doing a lot of work here. It's the multiple-choice MBE portion, not the full bar. Real bar exams include essays and performance tests that require sustained legal reasoning — a very different capability.

"DeepSeek-R1-Zero naturally learns to solve reasoning tasks with more thinking time through self-evolution"

The 'aha moment' — where the model spontaneously learns to backtrack and self-verify during training — reads like an emergent capability story. But it's worth noting this emerged in a heavily constrained setting: verifiable domains with binary rewards. Whether this self-evolution generalizes to open-ended reasoning without ground truth is the open question the paper explicitly declines to answer.

"a missing principle is making attention algorithms IO-aware—accounting for reads and writes between levels of GPU memory"

IO-awareness had existed as a concept in HPC for decades before FlashAttention applied it to transformers. The contribution is recognizing that attention's bottleneck is memory bandwidth, not arithmetic — a non-obvious insight given that attention is presented as an O(n²) compute problem. This reframing changed how researchers think about transformer optimization: FLOP counts are irrelevant if you're memory-bound, which most inference workloads are.

"Making language models bigger does not inherently make them better at following a user's intent."

This sentence launched a paradigm shift but contains a subtle conflation: 'following intent' and 'being capable' are different properties. Bigger models are better at capabilities; alignment is a separate axis. InstructGPT showed that small models with RLHF can beat large models without it on human preference ratings — but human preference ratings are not the same as actually doing what users want. The finding is real; the framing elides what 'better' means.

"emergent abilities appear due the researcher's choice of metric rather than due to fundamental changes in model behavior with scale"

This is one of the most important methodological critiques in NLP in the last five years. If 'emergence' is an artifact of discontinuous metrics, then the claim that scaling produces qualitatively new capabilities is a measurement illusion — models are just getting uniformly better, and we're quantizing that improvement into apparent phase transitions. The implication for safety research is uncomfortable: discontinuous capability jumps may be much rarer than believed.

"Humans can generally perform a new language task from only a few examples or from simple instructions – something which current NLP systems still largely struggle to do."

This framing positioned GPT-3 as closing the gap with human generalization, but 'struggling' carries more freight than acknowledged. GPT-3 still fails on tasks a child handles trivially — counting objects, spatial reasoning, tracking referents across long contexts. The paper conflates benchmark performance with cognitive generalization in a way that influenced how the field measured 'intelligence' for years.