Coalescent theory is a retrospective model of population genetics that traces the lineage of a sample of alleles back to their most recent common ancestor. It provides a framework for understanding the genetic diversity in a population by considering the genealogical history of alleles under various evolutionary forces like mutation, selection, and genetic drift.