A Gauss sum is a type of finite sum involving exponential functions and is a crucial tool in number theory, particularly in understanding quadratic residues and non-residues. These sums are used to establish deep connections between number theory and other areas of mathematics, such as representation theory and algebraic geometry.