The self-reference paradox arises when a statement refers to itself in a way that creates a logical contradiction, such as the classic 'This statement is false.' It challenges foundational principles in logic and mathematics by highlighting limitations in formal systems and the potential for inconsistency when self-reference is involved.