Orthogonal matrices are square matrices whose rows and columns are orthogonal unit vectors, and they preserve vector norms and angles when applied as transformations. They have the property that their transpose is equal to their inverse, making them fundamental in numerical stability and simplifying many linear algebra problems.