\alpha

Alpha

Naming convention: meaning depends on the field

In plain English

A conventional name for "a parameter that matters". Most often in this curriculum, $\alpha$ is a power-law exponent, as in $L \propto N^{-\alpha}$ , where it tells you how fast loss falls as models grow. In statistics $\alpha$ is the significance threshold (often 0.05). Older ML texts use $\alpha$ for the learning rate. In physics $\alpha \approx 1/137$ is the fine-structure constant. Different papers, different alphas.

See it

exponent1.35

Exponents such as

\alpha

and

\beta

set how steeply one quantity responds to another,

y = x^{\pm\alpha}

What it means, field by field

AI / ML

Scaling-law exponents (α_N, α_D, α_C in Kaplan et al. 2020); sometimes the learning rate; attention weights α_ij.

Physics

Fine-structure constant α ≈ 1/137; critical exponents near phase transitions; angular acceleration.

Biology

Allometric exponents (B ∝ M^α); alpha diversity in ecology.

Epistemics

Significance level α: the false-positive rate a test tolerates.

In a research paper

Kaplan et al. 2020, "Scaling Laws for Neural Language Models" (arXiv:2001.08361): test loss vs non-embedding parameter count.

L(N) = \left(\frac{N_c}{N}\right)^{\alpha_N},\quad \alpha_N \approx 0.076

Loss falls as a power of model size. A small α means slow but steady improvement over many orders of magnitude.

Common misreadings

Treating a fitted α as a law of nature. Scaling exponents are measured within a range and setup; Hoffmann et al. 2022 revised the compute-optimal allocation.
Confusing α as exponent with α as significance level in the same reading session.

Connected symbols

\propto

Proportional to

\beta

Beta

\log

Logarithm

\lambda

Lambda

Where it appears on this site

∂ An illustrative neural scaling law ∂ Kleiber’s law (metabolic scaling)§ 1. Learning to Read Equations § 5. Complexity and Systems Thinking § 7. The Physics–Biology–AI Connection Lab