A residue system is a complete set of representatives for the equivalence classes of integers modulo n, providing a foundational structure in number theory for solving congruences. It simplifies complex arithmetic operations by reducing them to a smaller, manageable set of integers, facilitating computations in modular arithmetic and cryptography.